Sep 19 09:06:26.205390 ip-10-0-131-218 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 09:06:26.636084 ip-10-0-131-218 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 09:06:26.636084 ip-10-0-131-218 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 09:06:26.636084 ip-10-0-131-218 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 09:06:26.636084 ip-10-0-131-218 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 09:06:26.636084 ip-10-0-131-218 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 09:06:26.638009 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.637922    2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 09:06:26.642068 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642051    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 09:06:26.642068 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642068    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642073    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642077    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642080    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642083    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642085    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642091    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642106    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642110    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642113    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642116    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642119    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642123    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642126    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642129    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642132    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642135    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642139    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642143    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 09:06:26.642146 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642146    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642148    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642152    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642155    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642158    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642161    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642163    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642166    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642169    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642171    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642174    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642177    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642179    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642182    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642184    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642187    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642189    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642192    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642196    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642198    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 09:06:26.642685 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642201    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642203    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642206    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642209    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642212    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642214    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642216    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642219    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642221    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642224    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642226    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642228    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642231    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642233    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642237    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642240    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642242    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642245    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642247    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642250    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 09:06:26.643188 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642253    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642255    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642258    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642260    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642263    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642265    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642268    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642270    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642274    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642276    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642279    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642282    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642284    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642287    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642289    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642292    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642294    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642297    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642299    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642302    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 09:06:26.643667 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642304    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642307    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642309    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642312    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642315    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642319    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642322    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642325    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642328    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642330    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642333    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642336    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642338    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642341    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642343    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642346    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642348    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642351    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.642353    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643157    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 09:06:26.644172 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643165    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643168    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643172    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643175    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643178    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643181    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643183    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643186    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643189    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643191    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643194    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643197    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643199    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643202    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643204    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643207    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643211    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643213    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643218    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643221    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 09:06:26.644679 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643223    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643226    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643228    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643231    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643233    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643236    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643238    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643240    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643243    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643245    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643248    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643250    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643253    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643255    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643258    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643260    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643263    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643267    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643271    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643274    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 09:06:26.645229 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643276    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643278    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643281    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643285    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643287    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643289    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643292    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643294    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643296    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643299    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643303    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643305    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643308    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643310    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643313    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643315    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643318    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643320    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643323    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643325    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 09:06:26.645715 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643328    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643330    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643332    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643335    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643337    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643340    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643342    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643345    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643347    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643350    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643353    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643355    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643359    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643361    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643363    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643367    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643370    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643375    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643378    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 09:06:26.646219 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643381    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643384    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643387    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643390    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643393    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643395    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643398    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643401    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643403    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643407    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643409    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643411    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643414    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643416    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643419    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643421    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643424    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643426    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643429    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643823    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 09:06:26.646673 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643828    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643831    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643834    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643838    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643840    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643843    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643846    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643849    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643851    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643854    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643856    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643859    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643861    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643865    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643869    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643873    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643876    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643878    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643881    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 09:06:26.647171 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643883    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643886    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643888    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643891    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643893    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643896    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643899    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643901    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643904    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643906    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643909    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643911    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643914    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643916    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643919    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643921    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643924    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643927    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643930    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643932    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 09:06:26.647630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643935    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643938    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643940    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643943    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643945    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643947    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643950    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643952    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643955    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643957    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643960    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643963    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643965    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643968    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643970    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643973    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643975    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643978    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643980    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643982    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 09:06:26.648124 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643985    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643987    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643990    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643992    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643994    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643997    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.643999    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644002    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644004    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644008    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644012    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644018    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644021    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644024    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644027    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644030    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644033    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644036    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644038    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644041    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 09:06:26.648616 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644044    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644047    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644049    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644052    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644054    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644057    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644059    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644062    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644064    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644067    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644069    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644071    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644074    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644076    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644079    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644081    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644084    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644087    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644090    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644174    2581 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 09:06:26.649111 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644210    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644216    2581 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644220    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644230    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644233    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644238    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644242    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644245    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644248    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644253    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644257    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644259    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644263    2581 flags.go:64] FLAG: --cgroup-root=""
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644266    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644269    2581 flags.go:64] FLAG: --client-ca-file=""
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644272    2581 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644274    2581 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644279    2581 flags.go:64] FLAG: --cluster-domain=""
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644282    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644285    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644288    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644291    2581 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644295    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 09:06:26.649584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644298    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644301    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644304    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644307    2581 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644309    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644312    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644315    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644318    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644322    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644325    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644328    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644331    2581 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644335    2581 flags.go:64] FLAG: --enable-server="true"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644338    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644343    2581 flags.go:64] FLAG: --event-burst="100"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644345    2581 flags.go:64] FLAG: --event-qps="50"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644348    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644351    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644355    2581 flags.go:64] FLAG: --eviction-hard=""
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644360    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644363    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644366    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644376    2581 flags.go:64] FLAG: --eviction-soft=""
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644380    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644383    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 09:06:26.650179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644386    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644389    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644392    2581 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644394    2581 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644397    2581 flags.go:64] FLAG: --feature-gates=""
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644401    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644404    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644407    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644410    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644413    2581 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644416    2581 flags.go:64] FLAG: --help="false"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644419    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644422    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644425    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644428    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644431    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644434    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644437    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644439    2581 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644445    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644448    2581 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644451    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644455    2581 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644457    2581 flags.go:64] FLAG: --kube-reserved=""
Sep 19 09:06:26.650785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644460    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644464    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644467    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644470    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644473    2581 flags.go:64] FLAG: --lock-file=""
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644476    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644479    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644482    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644487    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644490    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644493    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644496    2581 flags.go:64] FLAG: --logging-format="text"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644498    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644501    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644504    2581 flags.go:64] FLAG: --manifest-url=""
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644507    2581 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644512    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644514    2581 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644518    2581 flags.go:64] FLAG: --max-pods="110"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644521    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644524    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644527    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644530    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644533    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644535    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 09:06:26.651392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644539    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644546    2581 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644550    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644553    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644556    2581 flags.go:64] FLAG: --pod-cidr=""
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644560    2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644566    2581 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644568    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644572    2581 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644575    2581 flags.go:64] FLAG: --port="10250"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644578    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644581    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f3374d929c83f516"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644584    2581 flags.go:64] FLAG: --qos-reserved=""
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644587    2581 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644590    2581 flags.go:64] FLAG: --register-node="true"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644593    2581 flags.go:64] FLAG: --register-with-taints=""
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644596    2581 flags.go:64] FLAG: --registry-burst="10"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644599    2581 flags.go:64] FLAG: --registry-qps="5"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644602    2581 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644604    2581 flags.go:64] FLAG: --reserved-memory=""
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644608    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644611    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644614    2581 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644617    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644620    2581 flags.go:64] FLAG: --runonce="false"
Sep 19 09:06:26.651987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644623    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644626    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644629    2581 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644631    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644634    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644637    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644640    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644643    2581 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644646    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644649    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644652    2581 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644655    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644658    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644661    2581 flags.go:64] FLAG: --system-cgroups=""
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644664    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644670    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644673    2581 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644676    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644681    2581 flags.go:64] FLAG: --tls-min-version=""
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644684    2581 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644687    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644690    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644692    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644695    2581 flags.go:64] FLAG: --v="2"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644699    2581 flags.go:64] FLAG: --version="false"
Sep 19 09:06:26.652628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644706    2581 flags.go:64] FLAG: --vmodule=""
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644711    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.644714    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644849    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644853    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644856    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644859    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644861    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644864    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644867    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644870    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644872    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644875    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644877    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644880    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644883    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644887    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644891    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644894    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 09:06:26.653283 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644897    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644900    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644903    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644908    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644911    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644913    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644915    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644918    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644920    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644923    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644925    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644927    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644930    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644933    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644936    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644940    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644943    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644945    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644948    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644951    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 09:06:26.653773 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644953    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644956    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644958    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644960    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644963    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644965    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644968    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644971    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644975    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644977    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644980    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644984    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644987    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644989    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644992    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644996    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.644999    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645002    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645004    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 09:06:26.654319 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645007    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645009    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645012    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645014    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645016    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645019    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645021    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645024    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645026    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645029    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645031    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645033    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645036    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645038    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645041    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645043    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645046    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645048    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645051    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645053    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 09:06:26.654787 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645056    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645058    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645060    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645063    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645066    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645068    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645071    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645073    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645077    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645081    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645084    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645087    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645089    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645091    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645094    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645111    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645113    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645116    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645118    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645121    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 09:06:26.655294 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645123    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645126    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645129    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.645132    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.645855    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.654802    2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.654817    2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.654839    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.654846    2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654906    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654910    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654913    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654916    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654919    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654922    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 09:06:26.655783 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654925    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654928    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654931    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654934    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654937    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654940    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654943    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654946    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654949    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654951    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654954    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654956    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654959    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654962    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654964    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654967    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654969    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654972    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654974    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654977    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 09:06:26.656179 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654980    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654982    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654986    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654991    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654994    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654996    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.654999    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655002    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655004    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655007    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655009    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655012    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655014    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655017    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655020    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655022    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655025    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655028    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655031    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 09:06:26.656665 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655033    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655036    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655039    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655041    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655044    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655046    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655049    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655051    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655054    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655056    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655059    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655061    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655063    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655066    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655068    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655070    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655073    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655076    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655079    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655082    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 09:06:26.657167 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655084    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655086    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655089    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655092    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655094    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655114    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655116    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655119    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655122    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655124    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655128    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655131    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655134    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655137    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655139    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655142    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655144    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655147    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655151    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 09:06:26.657654 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655154    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655157    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655159    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655162    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655164    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655167    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655170    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655172    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655176    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655179    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655182    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655186    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655189    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655192    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 09:06:26.658252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655194    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.655199    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655314    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655320    2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655323    2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655326    2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655329    2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655332    2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655335    2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655338    2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655341    2581 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655344    2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655346    2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655349    2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 09:06:26.658626 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655351    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655354    2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655356    2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655359    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655361    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655364    2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655366    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655369    2581 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655371    2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655373    2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655376    2581 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655378    2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655381    2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655383    2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655386    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655389    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655392    2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655395    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655397    2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655400    2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 09:06:26.658969 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655402    2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655405    2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655407    2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655409    2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655412    2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655414    2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655417    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655420    2581 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655423    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655426    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655429    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655431    2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655433    2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655436    2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655439    2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655443    2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655445    2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655448    2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655450    2581 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655453    2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 09:06:26.659488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655455    2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655458    2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655460    2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655464    2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655467    2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655470    2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655472    2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655475    2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655478    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655482    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655485    2581 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655487    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655490    2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655493    2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655496    2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655498    2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655500    2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655503    2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655506    2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 09:06:26.659996 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655509    2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655511    2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655513    2581 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655517    2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655520    2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655523    2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655526    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655529    2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655531    2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655534    2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655537    2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655540    2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655542    2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655545    2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655548    2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655551    2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655553    2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655556    2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655558    2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655561    2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 09:06:26.660466 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655565    2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655568    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655571    2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655574    2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655577    2581 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655579    2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655582    2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:26.655585    2581 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.655590    2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.655705    2581 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.658179    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.659284    2581 server.go:1019] "Starting client certificate rotation"
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.659382    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 09:06:26.660960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.659415    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 09:06:26.665281 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.665254    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 09:06:26.668054 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.668032    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 09:06:26.678088 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.678068    2581 log.go:25] "Validated CRI v1 runtime API"
Sep 19 09:06:26.679940 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.679922    2581 log.go:25] "Validated CRI v1 image API"
Sep 19 09:06:26.680801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.680778    2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 09:06:26.684218 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.684194    2581 fs.go:135] Filesystem UUIDs: map[2eb7cd5b-0faf-4e90-855c-707d39db65b8:/dev/nvme0n1p3 7064afcb-dfc6-4005-a6df-fb9735a3a9ea:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 19 09:06:26.684268 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.684219    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 09:06:26.690621 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.690510    2581 manager.go:216] Machine: {Timestamp:2026-09-19 09:06:26.689043409 +0000 UTC m=+0.375848698 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097237 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec234ade91e7aa48354bb2909a116bc1 SystemUUID:ec234ade-91e7-aa48-354b-b2909a116bc1 BootID:e60ecd09-1426-41aa-a755-f694c38f8538 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dd:82:00:b1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dd:82:00:b1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:c3:25:51:19:36 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 09:06:26.690621 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.690617    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 09:06:26.690724 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.690701    2581 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 09:06:26.694619 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.694594    2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 09:06:26.694768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.694625    2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-218.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 09:06:26.695451 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.695441    2581 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 09:06:26.695489 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.695454    2581 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 09:06:26.695489 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.695463    2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 09:06:26.696281 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.696271    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 09:06:26.696363 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.696354    2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 09:06:26.700999 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.700985    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 09:06:26.701146 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701137    2581 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 09:06:26.701177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701172    2581 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 09:06:26.701218 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701187    2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 09:06:26.701218 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701212    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 09:06:26.701272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701224    2581 kubelet.go:398] "Adding apiserver pod source"
Sep 19 09:06:26.701272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701235    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 09:06:26.701685 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701670    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 09:06:26.701751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.701697    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 09:06:26.704896 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.704869    2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 09:06:26.705220 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.705198    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 09:06:26.708481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.708463    2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 09:06:26.708553 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.708495    2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 09:06:26.709977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.709965    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 09:06:26.710020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.709981    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 09:06:26.710020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.709987    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 09:06:26.710020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.709992    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 09:06:26.710020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710000    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 09:06:26.710020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710005    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 09:06:26.710020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710018    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 09:06:26.710229 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710023    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 09:06:26.710229 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710030    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 09:06:26.710229 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710035    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 09:06:26.710229 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710043    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 09:06:26.710229 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.710048    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 09:06:26.711734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.711721    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 09:06:26.711734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.711732    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 09:06:26.715749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.715736    2581 server.go:1268] "Started kubelet"
Sep 19 09:06:26.715847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.715816    2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 09:06:26.715884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.715827    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 09:06:26.715912 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.715900    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 09:06:26.716251 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.716232    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-218.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 09:06:26.716457 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.716432    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 09:06:26.716541 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.716530    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 09:06:26.716678 ip-10-0-131-218 systemd[1]: Started Kubernetes Kubelet.
Sep 19 09:06:26.717497 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.717479    2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 09:06:26.718413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.718401    2581 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 09:06:26.721373 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.721346    2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 09:06:26.726927 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.726900    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 09:06:26.727643 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.727626    2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 09:06:26.727779 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.727724    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:26.728576 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.728555    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 09:06:26.728576 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.728564    2581 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 09:06:26.728755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.728592    2581 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 09:06:26.728755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.728701    2581 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 09:06:26.728755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.728706    2581 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 09:06:26.730877 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.729753    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd4100610  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.715706896 +0000 UTC m=+0.402512184,LastTimestamp:2026-09-19 09:06:26.715706896 +0000 UTC m=+0.402512184,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.730967 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.730864    2581 factory.go:55] Registering systemd factory
Sep 19 09:06:26.730967 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.730893    2581 factory.go:223] Registration of the systemd container factory successfully
Sep 19 09:06:26.731148 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.731136    2581 factory.go:153] Registering CRI-O factory
Sep 19 09:06:26.731148 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.731148    2581 factory.go:223] Registration of the crio container factory successfully
Sep 19 09:06:26.731214 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.731189    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 09:06:26.731214 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.731210    2581 factory.go:103] Registering Raw factory
Sep 19 09:06:26.731266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.731224    2581 manager.go:1196] Started watching for new ooms in manager
Sep 19 09:06:26.731586 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.731575    2581 manager.go:349] Starting recovery of all containers
Sep 19 09:06:26.742333 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.742310    2581 manager.go:354] Recovery completed
Sep 19 09:06:26.747471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.747451    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:26.747683 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.747652    2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 09:06:26.747683 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.747666    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 09:06:26.751699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.751677    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:26.751801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.751711    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:26.751801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.751728    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:26.752357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.752328    2581 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 09:06:26.752357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.752339    2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 09:06:26.752357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.752354    2581 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 09:06:26.752527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.752395    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:26.753122 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.753078    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:26.753200 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.753130    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:26.753200 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.753146    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:26.754510 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.754497    2581 policy_none.go:49] "None policy: Start"
Sep 19 09:06:26.754552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.754513    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 09:06:26.754552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.754523    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 09:06:26.756516 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.756504    2581 policy_none.go:47] "Start"
Sep 19 09:06:26.767511 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.767435    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd635315a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751697242 +0000 UTC m=+0.438502531,LastTimestamp:2026-09-19 09:06:26.751697242 +0000 UTC m=+0.438502531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.769517 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.769446    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd63594ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751722734 +0000 UTC m=+0.438528024,LastTimestamp:2026-09-19 09:06:26.751722734 +0000 UTC m=+0.438528024,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.781595    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd635babf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751732415 +0000 UTC m=+0.438537705,LastTimestamp:2026-09-19 09:06:26.751732415 +0000 UTC m=+0.438537705,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.795704    2581 manager.go:337] "Starting Device Plugin manager"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.795787    2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.795798    2581 server.go:87] "Starting device plugin registration server"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.796046    2581 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.796056    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.796177    2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.796254    2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.796262    2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.796863    2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.796915    2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:26.805302 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.800500    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-218.ec2.internal.18d6ad9bd635315a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd635315a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751697242 +0000 UTC m=+0.438502531,LastTimestamp:2026-09-19 09:06:26.753118904 +0000 UTC m=+0.439924195,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.823058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.823019    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 09:06:26.824306 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.824293    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 09:06:26.824391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.824341    2581 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 09:06:26.824391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.824364    2581 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 09:06:26.824479 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.824417    2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 09:06:26.834818 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.834740    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-218.ec2.internal.18d6ad9bd63594ee\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd63594ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751722734 +0000 UTC m=+0.438528024,LastTimestamp:2026-09-19 09:06:26.753137748 +0000 UTC m=+0.439943037,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.834936 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.834883    2581 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass"
Sep 19 09:06:26.845605 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.845535    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-218.ec2.internal.18d6ad9bd635babf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd635babf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751732415 +0000 UTC m=+0.438537705,LastTimestamp:2026-09-19 09:06:26.753152698 +0000 UTC m=+0.439957991,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.867311 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.867240    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd8f5396b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.797836651 +0000 UTC m=+0.484641930,LastTimestamp:2026-09-19 09:06:26.797836651 +0000 UTC m=+0.484641930,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.893938 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.893890    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9p5w5"
Sep 19 09:06:26.896971 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.896947    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:26.897871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.897855    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:26.897952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.897890    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:26.897952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.897905    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:26.897952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.897927    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.912044 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.911968    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-218.ec2.internal.18d6ad9bd635315a\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd635315a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751697242 +0000 UTC m=+0.438502531,LastTimestamp:2026-09-19 09:06:26.897869849 +0000 UTC m=+0.584675139,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.914784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.914769    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9p5w5"
Sep 19 09:06:26.922992 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.922928    2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-218.ec2.internal.18d6ad9bd63594ee\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-218.ec2.internal.18d6ad9bd63594ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-218.ec2.internal,UID:ip-10-0-131-218.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-218.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-218.ec2.internal,},FirstTimestamp:2026-09-19 09:06:26.751722734 +0000 UTC m=+0.438528024,LastTimestamp:2026-09-19 09:06:26.897898006 +0000 UTC m=+0.584703295,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-218.ec2.internal,}"
Sep 19 09:06:26.923170 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.923151    2581 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-131-218.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.924527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.924510    2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"]
Sep 19 09:06:26.924580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.924571    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:26.925418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.925398    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:26.925495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.925432    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:26.925495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.925448    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:26.927763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.927751    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:26.927861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.927847    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.927914 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.927876    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:26.928478 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.928463    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:26.928550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.928495    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:26.928550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.928506    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:26.928550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.928465    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:26.928550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.928531    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:26.928550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.928540    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:26.929464 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.929448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7e4c44a51bdfd73ff16543061b2fc92-config\") pod \"kube-apiserver-proxy-ip-10-0-131-218.ec2.internal\" (UID: \"b7e4c44a51bdfd73ff16543061b2fc92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.929537 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.929470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bcd3183000776b215ef47da24fc6ec3d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"bcd3183000776b215ef47da24fc6ec3d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.929537 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.929487    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd3183000776b215ef47da24fc6ec3d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"bcd3183000776b215ef47da24fc6ec3d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.930896 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.930881    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.930969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.930917    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:26.931659 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.931642    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:26.931745 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.931672    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:26.931745 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:26.931681    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:26.946621 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.946603    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-218.ec2.internal\" not found" node="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.950544 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.950527    2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-218.ec2.internal\" not found" node="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:26.963329 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:26.963305    2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-218.ec2.internal\" not found" node="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.030424 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.030396    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd3183000776b215ef47da24fc6ec3d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"bcd3183000776b215ef47da24fc6ec3d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.030424 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.030426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7e4c44a51bdfd73ff16543061b2fc92-config\") pod \"kube-apiserver-proxy-ip-10-0-131-218.ec2.internal\" (UID: \"b7e4c44a51bdfd73ff16543061b2fc92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.030554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.030441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bcd3183000776b215ef47da24fc6ec3d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"bcd3183000776b215ef47da24fc6ec3d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.030554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.030466    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bcd3183000776b215ef47da24fc6ec3d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"bcd3183000776b215ef47da24fc6ec3d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.030554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.030498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bcd3183000776b215ef47da24fc6ec3d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal\" (UID: \"bcd3183000776b215ef47da24fc6ec3d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.030554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.030498    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7e4c44a51bdfd73ff16543061b2fc92-config\") pod \"kube-apiserver-proxy-ip-10-0-131-218.ec2.internal\" (UID: \"b7e4c44a51bdfd73ff16543061b2fc92\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.123576 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.123530    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:27.124740 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.124723    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:27.124825 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.124754    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:27.124825 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.124769    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:27.124825 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.124801    2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.135925 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.135904    2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.136002 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.135927    2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-218.ec2.internal\": node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.184798 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.184722    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.250422 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.250393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.254244 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.254225    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:27.285531 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.285501    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.386073 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.386029    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.486660 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.486595    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.565452 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.565426    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 09:06:27.587449 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.587421    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.658930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.658898    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 09:06:27.659383 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.659051    2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 09:06:27.688204 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.688172    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.788651 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.788452    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.788821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.788805    2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 09:06:27.889174 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.889134    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:27.921324 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.921284    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 09:01:27 +0000 UTC" deadline="2028-03-23 12:38:30.499402528 +0000 UTC"
Sep 19 09:06:27.921324 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.921319    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13227h32m2.578087333s"
Sep 19 09:06:27.923447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.923420    2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 09:06:27.924775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.924758    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 09:06:27.924888 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.924789    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 09:06:27.924888 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.924800    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeHasSufficientPID"
Sep 19 09:06:27.947536 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:27.947494    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd3183000776b215ef47da24fc6ec3d.slice/crio-dba166851e8b6840643fd5a8eba6dadb957b1352c158ba9f707533419ef80c0b WatchSource:0}: Error finding container dba166851e8b6840643fd5a8eba6dadb957b1352c158ba9f707533419ef80c0b: Status 404 returned error can't find the container with id dba166851e8b6840643fd5a8eba6dadb957b1352c158ba9f707533419ef80c0b
Sep 19 09:06:27.947939 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:27.947916    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e4c44a51bdfd73ff16543061b2fc92.slice/crio-a085d7940c91f2cf031397d80c26dcf42d7ac4bb9142e8e42d1e9c97a26c02d1 WatchSource:0}: Error finding container a085d7940c91f2cf031397d80c26dcf42d7ac4bb9142e8e42d1e9c97a26c02d1: Status 404 returned error can't find the container with id a085d7940c91f2cf031397d80c26dcf42d7ac4bb9142e8e42d1e9c97a26c02d1
Sep 19 09:06:27.953215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:27.953201    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 09:06:27.989555 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:27.989517    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:28.027158 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.027130    2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 09:06:28.090780 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.090696    2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-218.ec2.internal\" not found"
Sep 19 09:06:28.177736 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.177707    2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 09:06:28.229850 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.229811    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:28.248412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.248387    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 09:06:28.249526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.249512    2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal"
Sep 19 09:06:28.269226 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.269201    2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 09:06:28.702566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.702525    2581 apiserver.go:52] "Watching apiserver"
Sep 19 09:06:28.712956 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.712928    2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 09:06:28.713436 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.713404    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-592rd","kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9","openshift-cluster-node-tuning-operator/tuned-ptwb2","openshift-image-registry/node-ca-jjlx7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal","openshift-multus/multus-additional-cni-plugins-hc2pd","openshift-multus/network-metrics-daemon-485pv","kube-system/global-pull-secret-syncer-kr42v","openshift-dns/node-resolver-4g6kn","openshift-multus/multus-g6pzw","openshift-network-diagnostics/network-check-target-nl6b5","openshift-network-operator/iptables-alerter-5zt8t","openshift-ovn-kubernetes/ovnkube-node-vk7xn"]
Sep 19 09:06:28.716292 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.716269    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.718584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.718563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.719129 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.719083    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 09:06:28.721027 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.720996    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.723525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.723497    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.724266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.724239    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 09:06:28.724362 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.724302    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 09:06:28.724410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.724392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fs6mh\""
Sep 19 09:06:28.724410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.724246    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 09:06:28.725879 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.725852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.726373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.726353    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 09:06:28.726373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.726365    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 09:06:28.726542 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.726427    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tqk6r\""
Sep 19 09:06:28.726778 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.726760    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 09:06:28.727124 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.727084    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 09:06:28.727261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.727247    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 09:06:28.727391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.727367    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.728688    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cmxbx\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.728815    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.728924    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.728941    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.729033    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xcghg\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.729096    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.729163    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.729330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.729969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.729391    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nvxjc\""
Sep 19 09:06:28.730566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.730268    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 09:06:28.731502 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.731481    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.731603 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.731568    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:28.734414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.734392    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 09:06:28.735760 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.735692    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 09:06:28.736927 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.736662    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 09:06:28.736927 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.736813    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 09:06:28.737142 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.737119    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:28.738116 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.738075    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 09:06:28.738411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.738372    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cl85h\""
Sep 19 09:06:28.738511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.738474    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 09:06:28.738879 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.738833    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 09:06:28.739495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.739477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:28.739592 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.739545    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:28.741501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741477    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.741612 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741517    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysctl-d\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.741612 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741545    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d22aa59f-62ed-4687-a495-68cd7ba178aa-host-slash\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.741612 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741570    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-sys-fs\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.741612 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-host\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.741831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.741831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.741831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysconfig\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.741831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-kubernetes\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.741831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-systemd-units\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.741831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-log-socket\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.741831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741805    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-env-overrides\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-systemd\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqplk\" (UniqueName: \"kubernetes.io/projected/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-kube-api-access-fqplk\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741925    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-node-log\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-cni-bin\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.741998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-lib-modules\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742030    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60bb9f92-155a-4f69-b9d8-845edce54d11-dbus\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742075    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742129    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-run-netns\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742158    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60bb9f92-155a-4f69-b9d8-845edce54d11-kubelet-config\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.742235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742217    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742185    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-systemd\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742332    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-etc-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-cni-netd\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742400    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-cni-binary-copy\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkzv4\" (UniqueName: \"kubernetes.io/projected/3acf90b5-0bc3-4fff-b505-709199cad503-kube-api-access-wkzv4\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-serviceca\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742464    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-kubelet\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdfjd\" (UniqueName: \"kubernetes.io/projected/173d481c-5843-462c-b6fc-b645554ea972-kube-api-access-bdfjd\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-var-lib-kubelet\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d22aa59f-62ed-4687-a495-68cd7ba178aa-iptables-alerter-script\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742531    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-var-lib-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/14038d22-868b-4a2f-beb9-b1a287b970d6-ovn-node-metrics-cert\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742615    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f5zk\" (UniqueName: \"kubernetes.io/projected/14038d22-868b-4a2f-beb9-b1a287b970d6-kube-api-access-5f5zk\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742661    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742696    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-host\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.742898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-registration-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742755    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742781    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-run\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-ovnkube-script-lib\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-system-cni-dir\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysctl-conf\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742910    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-device-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742937    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlmqt\" (UniqueName: \"kubernetes.io/projected/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-kube-api-access-mlmqt\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-socket-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.742987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-slash\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743022    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-ovnkube-config\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-cnibin\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743086    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743126    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-modprobe-d\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743196    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-sys\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743222    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3acf90b5-0bc3-4fff-b505-709199cad503-tmp\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.743718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743247    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6968r\" (UniqueName: \"kubernetes.io/projected/d22aa59f-62ed-4687-a495-68cd7ba178aa-kube-api-access-6968r\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.744587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-ovn\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.744587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743320    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.744587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.743344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3acf90b5-0bc3-4fff-b505-709199cad503-etc-tuned\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.744842 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.744588    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.747172 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.747152    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:28.747278 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.747216    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.748699    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.748944    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.748946    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.749015    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5sslj\""
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.748946    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.749181    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-st6b2\""
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.749187    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mkfsk\""
Sep 19 09:06:28.749299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.749273    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 09:06:28.761074 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.761046    2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 09:06:28.830361 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.829755    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 09:06:28.835524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.835425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" event={"ID":"bcd3183000776b215ef47da24fc6ec3d","Type":"ContainerStarted","Data":"dba166851e8b6840643fd5a8eba6dadb957b1352c158ba9f707533419ef80c0b"}
Sep 19 09:06:28.837052 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.836987    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal" event={"ID":"b7e4c44a51bdfd73ff16543061b2fc92","Type":"ContainerStarted","Data":"a085d7940c91f2cf031397d80c26dcf42d7ac4bb9142e8e42d1e9c97a26c02d1"}
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/14038d22-868b-4a2f-beb9-b1a287b970d6-ovn-node-metrics-cert\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f5zk\" (UniqueName: \"kubernetes.io/projected/14038d22-868b-4a2f-beb9-b1a287b970d6-kube-api-access-5f5zk\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843642    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-cni-bin\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-host\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843696    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-registration-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-run\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c80ec8c6-a38b-496c-baaa-33d918287237-tmp-dir\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843803    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-ovnkube-script-lib\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-system-cni-dir\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysctl-conf\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843866    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-kubelet\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843882    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-device-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843906    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlmqt\" (UniqueName: \"kubernetes.io/projected/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-kube-api-access-mlmqt\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-cni-multus\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.847055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-daemon-config\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.843991    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-socket-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844016    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-slash\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-ovnkube-config\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844061    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-cnibin\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ecfb471-1cc3-414d-add4-29f6ce623c8c-konnectivity-ca\") pod \"konnectivity-agent-592rd\" (UID: \"6ecfb471-1cc3-414d-add4-29f6ce623c8c\") " pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844091    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-cnibin\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844106    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ddc969e-c89d-4345-a7a5-d901f9ebe551-cni-binary-copy\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-modprobe-d\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-sys\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3acf90b5-0bc3-4fff-b505-709199cad503-tmp\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844245    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-system-cni-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-socket-dir-parent\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-etc-kubernetes\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c80ec8c6-a38b-496c-baaa-33d918287237-hosts-file\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6968r\" (UniqueName: \"kubernetes.io/projected/d22aa59f-62ed-4687-a495-68cd7ba178aa-kube-api-access-6968r\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.848078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-ovn\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3acf90b5-0bc3-4fff-b505-709199cad503-etc-tuned\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-multus-certs\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844569    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysctl-d\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844594    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ecfb471-1cc3-414d-add4-29f6ce623c8c-agent-certs\") pod \"konnectivity-agent-592rd\" (UID: \"6ecfb471-1cc3-414d-add4-29f6ce623c8c\") " pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d22aa59f-62ed-4687-a495-68cd7ba178aa-host-slash\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-sys-fs\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-host\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844740    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysconfig\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-kubernetes\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-netns\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844950    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-systemd-units\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.844983    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-log-socket\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-env-overrides\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-systemd\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ff72\" (UniqueName: \"kubernetes.io/projected/0ddc969e-c89d-4345-a7a5-d901f9ebe551-kube-api-access-4ff72\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845094    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845120    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4q2g\" (UniqueName: \"kubernetes.io/projected/e0c101dc-58b0-481d-9859-234a79696788-kube-api-access-p4q2g\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqplk\" (UniqueName: \"kubernetes.io/projected/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-kube-api-access-fqplk\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-node-log\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-cni-bin\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845293    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-lib-modules\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60bb9f92-155a-4f69-b9d8-845edce54d11-dbus\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845390    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-run-netns\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845416    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60bb9f92-155a-4f69-b9d8-845edce54d11-kubelet-config\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-cni-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845508    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-systemd\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.849827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-etc-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845586    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-cni-netd\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-cni-binary-copy\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkzv4\" (UniqueName: \"kubernetes.io/projected/3acf90b5-0bc3-4fff-b505-709199cad503-kube-api-access-wkzv4\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845705    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-k8s-cni-cncf-io\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845731    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxv8x\" (UniqueName: \"kubernetes.io/projected/c80ec8c6-a38b-496c-baaa-33d918287237-kube-api-access-cxv8x\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845772    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-serviceca\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-kubelet\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdfjd\" (UniqueName: \"kubernetes.io/projected/173d481c-5843-462c-b6fc-b645554ea972-kube-api-access-bdfjd\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-var-lib-kubelet\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-hostroot\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-conf-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845956    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d22aa59f-62ed-4687-a495-68cd7ba178aa-iptables-alerter-script\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.845982    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-var-lib-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.846096    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-var-lib-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.846577    2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.846895    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:28.850455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.846971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-sys-fs\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.847016    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret podName:60bb9f92-155a-4f69-b9d8-845edce54d11 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:29.346993401 +0000 UTC m=+3.033798693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret") pod "global-pull-secret-syncer-kr42v" (UID: "60bb9f92-155a-4f69-b9d8-845edce54d11") : object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-host\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847066    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-host\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847200    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-registration-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847239    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847294    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-etc-selinux\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847313    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysconfig\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-kubernetes\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847471    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysctl-conf\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847549    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-device-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-socket-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.847989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-ovnkube-script-lib\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-system-cni-dir\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-slash\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.851411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848098    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-log-socket\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848109    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-systemd-units\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-env-overrides\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848534    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-systemd\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848769    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848802    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-node-log\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848838    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-cni-bin\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.848909    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-lib-modules\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/60bb9f92-155a-4f69-b9d8-845edce54d11-dbus\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849083    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849119    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-run-netns\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849168    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/60bb9f92-155a-4f69-b9d8-845edce54d11-kubelet-config\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-systemd\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849259    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-etc-openvswitch\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849293    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-cni-netd\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.849820    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-cni-binary-copy\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.850296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-kubelet\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.850526    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-serviceca\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.852242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.850579    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-var-lib-kubelet\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.851146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d22aa59f-62ed-4687-a495-68cd7ba178aa-iptables-alerter-script\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.851241    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-run\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.851242    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/173d481c-5843-462c-b6fc-b645554ea972-cnibin\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.846895    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d22aa59f-62ed-4687-a495-68cd7ba178aa-host-slash\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.851819    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.851873    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-run-ovn\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.851974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/173d481c-5843-462c-b6fc-b645554ea972-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.852092    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-sys\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.852390    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-modprobe-d\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.852483    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3acf90b5-0bc3-4fff-b505-709199cad503-etc-sysctl-d\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.852524    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/14038d22-868b-4a2f-beb9-b1a287b970d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.853115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.852686    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/14038d22-868b-4a2f-beb9-b1a287b970d6-ovnkube-config\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.854635 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.853976    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xm9l6"
Sep 19 09:06:28.854931 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.854908    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3acf90b5-0bc3-4fff-b505-709199cad503-tmp\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.856084 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.856061    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/14038d22-868b-4a2f-beb9-b1a287b970d6-ovn-node-metrics-cert\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.865921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.865889    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3acf90b5-0bc3-4fff-b505-709199cad503-etc-tuned\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.867903 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.867875    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlmqt\" (UniqueName: \"kubernetes.io/projected/f9f60cbc-401d-42fb-ad66-46b87ae25cc7-kube-api-access-mlmqt\") pod \"aws-ebs-csi-driver-node-8n5d9\" (UID: \"f9f60cbc-401d-42fb-ad66-46b87ae25cc7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:28.870404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.870373    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xm9l6"
Sep 19 09:06:28.875448 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.875421    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f5zk\" (UniqueName: \"kubernetes.io/projected/14038d22-868b-4a2f-beb9-b1a287b970d6-kube-api-access-5f5zk\") pod \"ovnkube-node-vk7xn\" (UID: \"14038d22-868b-4a2f-beb9-b1a287b970d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:28.876835 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.876745    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqplk\" (UniqueName: \"kubernetes.io/projected/10fdafc3-1641-4fa0-8ec1-3d403c30e9b5-kube-api-access-fqplk\") pod \"node-ca-jjlx7\" (UID: \"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5\") " pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:28.877078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.877055    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6968r\" (UniqueName: \"kubernetes.io/projected/d22aa59f-62ed-4687-a495-68cd7ba178aa-kube-api-access-6968r\") pod \"iptables-alerter-5zt8t\" (UID: \"d22aa59f-62ed-4687-a495-68cd7ba178aa\") " pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:28.878005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.877976    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkzv4\" (UniqueName: \"kubernetes.io/projected/3acf90b5-0bc3-4fff-b505-709199cad503-kube-api-access-wkzv4\") pod \"tuned-ptwb2\" (UID: \"3acf90b5-0bc3-4fff-b505-709199cad503\") " pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:28.879421 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.879379    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdfjd\" (UniqueName: \"kubernetes.io/projected/173d481c-5843-462c-b6fc-b645554ea972-kube-api-access-bdfjd\") pod \"multus-additional-cni-plugins-hc2pd\" (UID: \"173d481c-5843-462c-b6fc-b645554ea972\") " pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:28.946387 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-netns\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.946599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ff72\" (UniqueName: \"kubernetes.io/projected/0ddc969e-c89d-4345-a7a5-d901f9ebe551-kube-api-access-4ff72\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.946599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:28.946599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4q2g\" (UniqueName: \"kubernetes.io/projected/e0c101dc-58b0-481d-9859-234a79696788-kube-api-access-p4q2g\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:28.946599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-cni-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.946599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-k8s-cni-cncf-io\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.946599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxv8x\" (UniqueName: \"kubernetes.io/projected/c80ec8c6-a38b-496c-baaa-33d918287237-kube-api-access-cxv8x\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.946924 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-hostroot\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.946924 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-conf-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.946924 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-cni-bin\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.947074 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.946759    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-cni-bin\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.947590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.947563    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-cni-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.947690 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.947640    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-k8s-cni-cncf-io\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.947985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.947938    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-hostroot\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948067 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948030    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-conf-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948067 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948035    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c80ec8c6-a38b-496c-baaa-33d918287237-tmp-dir\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.948195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948110    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-kubelet\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948155    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-cni-multus\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948226    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-daemon-config\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948361 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948264    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ecfb471-1cc3-414d-add4-29f6ce623c8c-konnectivity-ca\") pod \"konnectivity-agent-592rd\" (UID: \"6ecfb471-1cc3-414d-add4-29f6ce623c8c\") " pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:28.948361 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-cnibin\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948467 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ddc969e-c89d-4345-a7a5-d901f9ebe551-cni-binary-copy\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948467 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948365    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-netns\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.948467 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.948458    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-cnibin\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949068    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ddc969e-c89d-4345-a7a5-d901f9ebe551-cni-binary-copy\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949090    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-daemon-config\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949150    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-system-cni-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-socket-dir-parent\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949251    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-etc-kubernetes\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.949259    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c80ec8c6-a38b-496c-baaa-33d918287237-hosts-file\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-multus-certs\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ecfb471-1cc3-414d-add4-29f6ce623c8c-agent-certs\") pod \"konnectivity-agent-592rd\" (UID: \"6ecfb471-1cc3-414d-add4-29f6ce623c8c\") " pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949575    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-multus-socket-dir-parent\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949639    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-kubelet\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-var-lib-cni-multus\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-etc-kubernetes\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949887    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c80ec8c6-a38b-496c-baaa-33d918287237-hosts-file\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-system-cni-dir\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.949979    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0ddc969e-c89d-4345-a7a5-d901f9ebe551-host-run-multus-certs\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:28.953734 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.950021    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:29.45000093 +0000 UTC m=+3.136806210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:28.954727 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.950616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6ecfb471-1cc3-414d-add4-29f6ce623c8c-konnectivity-ca\") pod \"konnectivity-agent-592rd\" (UID: \"6ecfb471-1cc3-414d-add4-29f6ce623c8c\") " pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:28.954727 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.950696    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c80ec8c6-a38b-496c-baaa-33d918287237-tmp-dir\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.955438 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.954926    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6ecfb471-1cc3-414d-add4-29f6ce623c8c-agent-certs\") pod \"konnectivity-agent-592rd\" (UID: \"6ecfb471-1cc3-414d-add4-29f6ce623c8c\") " pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:28.976684 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.976620    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4q2g\" (UniqueName: \"kubernetes.io/projected/e0c101dc-58b0-481d-9859-234a79696788-kube-api-access-p4q2g\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:28.982008 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.981748    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 09:06:28.982008 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.981778    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 09:06:28.982008 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.981794    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w5mc2 for pod openshift-network-diagnostics/network-check-target-nl6b5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:28.982008 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:28.981892    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2 podName:3da542d9-bdb7-460e-a8dc-da326cf2cec0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:29.481843741 +0000 UTC m=+3.168649037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w5mc2" (UniqueName: "kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2") pod "network-check-target-nl6b5" (UID: "3da542d9-bdb7-460e-a8dc-da326cf2cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:28.982872 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.982835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxv8x\" (UniqueName: \"kubernetes.io/projected/c80ec8c6-a38b-496c-baaa-33d918287237-kube-api-access-cxv8x\") pod \"node-resolver-4g6kn\" (UID: \"c80ec8c6-a38b-496c-baaa-33d918287237\") " pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:28.984742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:28.984717    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ff72\" (UniqueName: \"kubernetes.io/projected/0ddc969e-c89d-4345-a7a5-d901f9ebe551-kube-api-access-4ff72\") pod \"multus-g6pzw\" (UID: \"0ddc969e-c89d-4345-a7a5-d901f9ebe551\") " pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:29.029370 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.029334    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hc2pd"
Sep 19 09:06:29.040549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.040520    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5zt8t"
Sep 19 09:06:29.042357 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.042329    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173d481c_5843_462c_b6fc_b645554ea972.slice/crio-7a173815ba34b19d15cc8abf6157703f2afabc70fdc61ad831ab4db423a86af6 WatchSource:0}: Error finding container 7a173815ba34b19d15cc8abf6157703f2afabc70fdc61ad831ab4db423a86af6: Status 404 returned error can't find the container with id 7a173815ba34b19d15cc8abf6157703f2afabc70fdc61ad831ab4db423a86af6
Sep 19 09:06:29.048189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.048147    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9"
Sep 19 09:06:29.056079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.056060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ptwb2"
Sep 19 09:06:29.061039 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.060828    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f60cbc_401d_42fb_ad66_46b87ae25cc7.slice/crio-5d978c3be5c0539030070bff09c3ebb754d6e7e3cbde70c497b2586efd6ab124 WatchSource:0}: Error finding container 5d978c3be5c0539030070bff09c3ebb754d6e7e3cbde70c497b2586efd6ab124: Status 404 returned error can't find the container with id 5d978c3be5c0539030070bff09c3ebb754d6e7e3cbde70c497b2586efd6ab124
Sep 19 09:06:29.061938 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.061849    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jjlx7"
Sep 19 09:06:29.068815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.068794    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:29.069291 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.069262    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acf90b5_0bc3_4fff_b505_709199cad503.slice/crio-1d7287451dee55b7ac15c1eb1878d56deaf057265d825daacf226603e8f71ebb WatchSource:0}: Error finding container 1d7287451dee55b7ac15c1eb1878d56deaf057265d825daacf226603e8f71ebb: Status 404 returned error can't find the container with id 1d7287451dee55b7ac15c1eb1878d56deaf057265d825daacf226603e8f71ebb
Sep 19 09:06:29.083888 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.083862    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:29.086237 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.086205    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14038d22_868b_4a2f_beb9_b1a287b970d6.slice/crio-0d09c81fdfd2f595dbb0e54fc8118ff483193974d8e0f3976a3a223e8442f16e WatchSource:0}: Error finding container 0d09c81fdfd2f595dbb0e54fc8118ff483193974d8e0f3976a3a223e8442f16e: Status 404 returned error can't find the container with id 0d09c81fdfd2f595dbb0e54fc8118ff483193974d8e0f3976a3a223e8442f16e
Sep 19 09:06:29.087776 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.087646    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fdafc3_1641_4fa0_8ec1_3d403c30e9b5.slice/crio-2774028b1ede037d7fb1e71f84df604ea44cd7249aee6201bf1796f99ff9af69 WatchSource:0}: Error finding container 2774028b1ede037d7fb1e71f84df604ea44cd7249aee6201bf1796f99ff9af69: Status 404 returned error can't find the container with id 2774028b1ede037d7fb1e71f84df604ea44cd7249aee6201bf1796f99ff9af69
Sep 19 09:06:29.089386 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.089366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4g6kn"
Sep 19 09:06:29.097391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.097368    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g6pzw"
Sep 19 09:06:29.105772 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.105742    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ecfb471_1cc3_414d_add4_29f6ce623c8c.slice/crio-76e863bc8f14bb18309aaaba2ac4d4b7dd0d53d08f841ea7c410dbc71892ff28 WatchSource:0}: Error finding container 76e863bc8f14bb18309aaaba2ac4d4b7dd0d53d08f841ea7c410dbc71892ff28: Status 404 returned error can't find the container with id 76e863bc8f14bb18309aaaba2ac4d4b7dd0d53d08f841ea7c410dbc71892ff28
Sep 19 09:06:29.106752 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.106701    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80ec8c6_a38b_496c_baaa_33d918287237.slice/crio-021a08343d1f3da49059327ce5f8f3be24b85d2b9c5375982517276df7f92999 WatchSource:0}: Error finding container 021a08343d1f3da49059327ce5f8f3be24b85d2b9c5375982517276df7f92999: Status 404 returned error can't find the container with id 021a08343d1f3da49059327ce5f8f3be24b85d2b9c5375982517276df7f92999
Sep 19 09:06:29.112300 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:06:29.111905    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ddc969e_c89d_4345_a7a5_d901f9ebe551.slice/crio-1666035c2cdd96a4c6f3a5938ce31116ce46baac6340bfe713e8e860fd0f8add WatchSource:0}: Error finding container 1666035c2cdd96a4c6f3a5938ce31116ce46baac6340bfe713e8e860fd0f8add: Status 404 returned error can't find the container with id 1666035c2cdd96a4c6f3a5938ce31116ce46baac6340bfe713e8e860fd0f8add
Sep 19 09:06:29.352852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.352756    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:29.353032 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.352923    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:29.353032 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.353004    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret podName:60bb9f92-155a-4f69-b9d8-845edce54d11 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:30.352975636 +0000 UTC m=+4.039780929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret") pod "global-pull-secret-syncer-kr42v" (UID: "60bb9f92-155a-4f69-b9d8-845edce54d11") : object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:29.453824 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.453785    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:29.454014 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.453969    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:29.454070 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.454053    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:30.454032493 +0000 UTC m=+4.140837789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:29.554261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.554217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:29.554448 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.554369    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 09:06:29.554448 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.554391    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 09:06:29.554448 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.554402    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w5mc2 for pod openshift-network-diagnostics/network-check-target-nl6b5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:29.554599 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:29.554474    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2 podName:3da542d9-bdb7-460e-a8dc-da326cf2cec0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:30.554454211 +0000 UTC m=+4.241259500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w5mc2" (UniqueName: "kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2") pod "network-check-target-nl6b5" (UID: "3da542d9-bdb7-460e-a8dc-da326cf2cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:29.841977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.841924    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g6pzw" event={"ID":"0ddc969e-c89d-4345-a7a5-d901f9ebe551","Type":"ContainerStarted","Data":"1666035c2cdd96a4c6f3a5938ce31116ce46baac6340bfe713e8e860fd0f8add"}
Sep 19 09:06:29.843653 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.843591    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4g6kn" event={"ID":"c80ec8c6-a38b-496c-baaa-33d918287237","Type":"ContainerStarted","Data":"021a08343d1f3da49059327ce5f8f3be24b85d2b9c5375982517276df7f92999"}
Sep 19 09:06:29.850289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.850245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jjlx7" event={"ID":"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5","Type":"ContainerStarted","Data":"2774028b1ede037d7fb1e71f84df604ea44cd7249aee6201bf1796f99ff9af69"}
Sep 19 09:06:29.859299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.859270    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ptwb2" event={"ID":"3acf90b5-0bc3-4fff-b505-709199cad503","Type":"ContainerStarted","Data":"1d7287451dee55b7ac15c1eb1878d56deaf057265d825daacf226603e8f71ebb"}
Sep 19 09:06:29.861053 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.861031    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9" event={"ID":"f9f60cbc-401d-42fb-ad66-46b87ae25cc7","Type":"ContainerStarted","Data":"5d978c3be5c0539030070bff09c3ebb754d6e7e3cbde70c497b2586efd6ab124"}
Sep 19 09:06:29.863250 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.863228    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5zt8t" event={"ID":"d22aa59f-62ed-4687-a495-68cd7ba178aa","Type":"ContainerStarted","Data":"469aa088fabed3b57c9993fab024d54ebb163e91c8b097ed0f29928699ef5a0c"}
Sep 19 09:06:29.870058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.870032    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-592rd" event={"ID":"6ecfb471-1cc3-414d-add4-29f6ce623c8c","Type":"ContainerStarted","Data":"76e863bc8f14bb18309aaaba2ac4d4b7dd0d53d08f841ea7c410dbc71892ff28"}
Sep 19 09:06:29.873609 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.873586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"0d09c81fdfd2f595dbb0e54fc8118ff483193974d8e0f3976a3a223e8442f16e"}
Sep 19 09:06:29.875528 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.875480    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 09:01:29 +0000 UTC" deadline="2028-05-20 05:18:29.195441007 +0000 UTC"
Sep 19 09:06:29.875654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.875641    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14612h11m59.319805897s"
Sep 19 09:06:29.876133 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:29.876047    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerStarted","Data":"7a173815ba34b19d15cc8abf6157703f2afabc70fdc61ad831ab4db423a86af6"}
Sep 19 09:06:30.362884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.362097    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:30.362884 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.362417    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:30.362884 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.362488    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret podName:60bb9f92-155a-4f69-b9d8-845edce54d11 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:32.362466849 +0000 UTC m=+6.049272146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret") pod "global-pull-secret-syncer-kr42v" (UID: "60bb9f92-155a-4f69-b9d8-845edce54d11") : object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:30.444603 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.444573    2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 09:06:30.463205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.463161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:30.463455 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.463332    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:30.463535 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.463465    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:32.46344137 +0000 UTC m=+6.150246665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:30.564763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.564672    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:30.564948 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.564848    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 09:06:30.564948 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.564871    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 09:06:30.564948 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.564885    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w5mc2 for pod openshift-network-diagnostics/network-check-target-nl6b5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:30.564948 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.564948    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2 podName:3da542d9-bdb7-460e-a8dc-da326cf2cec0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:32.564928249 +0000 UTC m=+6.251733552 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w5mc2" (UniqueName: "kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2") pod "network-check-target-nl6b5" (UID: "3da542d9-bdb7-460e-a8dc-da326cf2cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:30.826089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.825986    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:30.826288 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.826195    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:30.826640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.826622    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:30.826763 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.826730    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:30.826838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.826812    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:30.826919 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:30.826896    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:30.876310 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.876261    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 09:01:29 +0000 UTC" deadline="2028-05-03 11:35:49.688194641 +0000 UTC"
Sep 19 09:06:30.876310 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:30.876309    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14210h29m18.811891027s"
Sep 19 09:06:31.886451 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:31.885425    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" event={"ID":"bcd3183000776b215ef47da24fc6ec3d","Type":"ContainerStarted","Data":"7b1abf5666ad5a618b440d172916a22438243faee27f723d4cf4a8088b59071c"}
Sep 19 09:06:31.896083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:31.895574    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal" event={"ID":"b7e4c44a51bdfd73ff16543061b2fc92","Type":"ContainerStarted","Data":"6535626e22c4eb19b6e2678cd88d0dcd897b0e217d79b7ffb67d54e585ae4301"}
Sep 19 09:06:31.920393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:31.919786    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-218.ec2.internal" podStartSLOduration=2.919764314 podStartE2EDuration="2.919764314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:31.919301449 +0000 UTC m=+5.606106750" watchObservedRunningTime="2026-09-19 09:06:31.919764314 +0000 UTC m=+5.606569617"
Sep 19 09:06:32.379596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:32.379550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:32.379786 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.379742    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:32.379866 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.379807    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret podName:60bb9f92-155a-4f69-b9d8-845edce54d11 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:36.37978847 +0000 UTC m=+10.066593752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret") pod "global-pull-secret-syncer-kr42v" (UID: "60bb9f92-155a-4f69-b9d8-845edce54d11") : object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:32.480015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:32.479970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:32.480219 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.480185    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:32.480306 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.480261    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:36.480239536 +0000 UTC m=+10.167044818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:32.580774 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:32.580732    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:32.581012 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.580994    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 09:06:32.581099 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.581021    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 09:06:32.581099 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.581035    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w5mc2 for pod openshift-network-diagnostics/network-check-target-nl6b5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:32.581256 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.581163    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2 podName:3da542d9-bdb7-460e-a8dc-da326cf2cec0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:36.581079618 +0000 UTC m=+10.267884915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w5mc2" (UniqueName: "kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2") pod "network-check-target-nl6b5" (UID: "3da542d9-bdb7-460e-a8dc-da326cf2cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:32.836423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:32.836320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:32.836589 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:32.836320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:32.836589 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:32.836324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:32.836589 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.836509    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:32.836746 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.836683    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:32.838486 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:32.837466    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:34.825413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:34.825376    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:34.825927 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:34.825384    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:34.825927 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:34.825532    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:34.825927 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:34.825589    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:34.825927 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:34.825705    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:34.825927 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:34.825858    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:36.413767 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:36.413719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:36.414305 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.413929    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:36.414305 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.414010    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret podName:60bb9f92-155a-4f69-b9d8-845edce54d11 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:44.413988728 +0000 UTC m=+18.100794014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret") pod "global-pull-secret-syncer-kr42v" (UID: "60bb9f92-155a-4f69-b9d8-845edce54d11") : object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:36.514689 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:36.514648    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:36.514878 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.514818    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:36.514947 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.514893    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:44.514870917 +0000 UTC m=+18.201676198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:36.616118 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:36.615441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:36.616118 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.615664    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 09:06:36.616118 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.615685    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 09:06:36.616118 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.615697    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w5mc2 for pod openshift-network-diagnostics/network-check-target-nl6b5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:36.616118 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.615757    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2 podName:3da542d9-bdb7-460e-a8dc-da326cf2cec0 nodeName:}" failed. No retries permitted until 2026-09-19 09:06:44.615737984 +0000 UTC m=+18.302543267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w5mc2" (UniqueName: "kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2") pod "network-check-target-nl6b5" (UID: "3da542d9-bdb7-460e-a8dc-da326cf2cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:36.826183 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:36.826089    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:36.826357 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.826231    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:36.826357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:36.826305    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:36.826472 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:36.826381    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:36.826472 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.826406    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:36.826472 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:36.826436    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:37.913582 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:37.913534    2581 generic.go:356] "Generic (PLEG): container finished" podID="bcd3183000776b215ef47da24fc6ec3d" containerID="7b1abf5666ad5a618b440d172916a22438243faee27f723d4cf4a8088b59071c" exitCode=0
Sep 19 09:06:37.913582 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:37.913586    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" event={"ID":"bcd3183000776b215ef47da24fc6ec3d","Type":"ContainerDied","Data":"7b1abf5666ad5a618b440d172916a22438243faee27f723d4cf4a8088b59071c"}
Sep 19 09:06:38.829484 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:38.829449    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:38.829655 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:38.829588    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:38.829996 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:38.829974    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:38.830101 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:38.830056    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:38.830179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:38.830125    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:38.830224 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:38.830173    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:40.825637 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:40.825599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:40.826089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:40.825599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:40.826089 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:40.825730    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:40.826089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:40.825599    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:40.826089 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:40.825825    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:40.826089 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:40.825883    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:42.825547 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:42.825508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:42.825963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:42.825508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:42.825963 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:42.825779    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:42.825963 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:42.825671    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:42.825963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:42.825808    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:42.825963 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:42.825879    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:44.481340 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:44.481297    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:44.481825 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.481429    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:44.481825 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.481483    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret podName:60bb9f92-155a-4f69-b9d8-845edce54d11 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:00.481467932 +0000 UTC m=+34.168273207 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret") pod "global-pull-secret-syncer-kr42v" (UID: "60bb9f92-155a-4f69-b9d8-845edce54d11") : object "kube-system"/"original-pull-secret" not registered
Sep 19 09:06:44.581942 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:44.581907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:44.582146 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.582082    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:44.582219 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.582180    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:00.582159697 +0000 UTC m=+34.268964974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:06:44.682720 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:44.682694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:44.682861 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.682846    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 09:06:44.682923 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.682869    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 09:06:44.682923 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.682881    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w5mc2 for pod openshift-network-diagnostics/network-check-target-nl6b5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:44.683004 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.682940    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2 podName:3da542d9-bdb7-460e-a8dc-da326cf2cec0 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:00.682923331 +0000 UTC m=+34.369728608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w5mc2" (UniqueName: "kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2") pod "network-check-target-nl6b5" (UID: "3da542d9-bdb7-460e-a8dc-da326cf2cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:06:44.824854 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:44.824802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:44.825022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:44.824802    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:44.825022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:44.824918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:44.825022 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.824915    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:44.825022 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.824981    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:44.825189 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:44.825092    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:46.826042 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:46.826010    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:46.826638 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:46.826109    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:46.826776 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:46.826659    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:46.826776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:46.826159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:46.826776 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:46.826736    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:46.826924 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:46.826826    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:48.829816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.826976    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:48.829816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.827390    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:48.829816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.827185    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:48.829816 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:48.827483    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:48.829816 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:48.827594    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:48.829816 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:48.827734    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:48.934726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.934509    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerStarted","Data":"5c710731fd8b0309d1ff6a5716227938f27e070ae5c3b7b53cd87cf039cc7e05"}
Sep 19 09:06:48.936950 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.936488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" event={"ID":"bcd3183000776b215ef47da24fc6ec3d","Type":"ContainerStarted","Data":"49a9ec6391e4ed5d14852fc81dd17240399b5059a17cbc2efcdb8446863ad9e3"}
Sep 19 09:06:48.938763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.938579    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g6pzw" event={"ID":"0ddc969e-c89d-4345-a7a5-d901f9ebe551","Type":"ContainerStarted","Data":"572b5bcd6fa5e0a8fa7c954e80f583859ece2d82ec9313abac52a7683601aca1"}
Sep 19 09:06:48.940706 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.940679    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4g6kn" event={"ID":"c80ec8c6-a38b-496c-baaa-33d918287237","Type":"ContainerStarted","Data":"fd1cc771a1c6dd2405967e3b96f526fbb8bab9202f5c48814b59334cbb871b28"}
Sep 19 09:06:48.941977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.941958    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jjlx7" event={"ID":"10fdafc3-1641-4fa0-8ec1-3d403c30e9b5","Type":"ContainerStarted","Data":"c0b60592199f8c70f8dd24c97b3d1cfb88de383a4f4991cc19526c6cc818e59d"}
Sep 19 09:06:48.943721 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.943693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ptwb2" event={"ID":"3acf90b5-0bc3-4fff-b505-709199cad503","Type":"ContainerStarted","Data":"71046cd35e3dfa362224c8f414b5de00554b9a0202bcf261bdbf91c00797a90f"}
Sep 19 09:06:48.945997 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.945973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9" event={"ID":"f9f60cbc-401d-42fb-ad66-46b87ae25cc7","Type":"ContainerStarted","Data":"f2cdfac1cbf6e85e11d9232c80ab8115a0e872d9b4686011f9bb8d560d3138c8"}
Sep 19 09:06:48.947291 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.947263    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-592rd" event={"ID":"6ecfb471-1cc3-414d-add4-29f6ce623c8c","Type":"ContainerStarted","Data":"3dffc97489e6de422af93aea20db702bd9ff8d803b3a851931f5a73247919858"}
Sep 19 09:06:48.949496 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.949471    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"3a83c5a8a4c351286db0109babd9524d24751de0975f68a09ab1da9865491d91"}
Sep 19 09:06:48.949576 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.949541    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"563c7c4474cab0b2d03803f81462bae80ffd1fa660ff86e733ab141bac945f5e"}
Sep 19 09:06:48.978018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:48.977956    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-218.ec2.internal" podStartSLOduration=19.97793375 podStartE2EDuration="19.97793375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:48.977332214 +0000 UTC m=+22.664137511" watchObservedRunningTime="2026-09-19 09:06:48.97793375 +0000 UTC m=+22.664739048"
Sep 19 09:06:49.004394 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.004348    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ptwb2" podStartSLOduration=1.437934534 podStartE2EDuration="21.004332129s" totalImagesPullingTime="19.566397595s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:49.003942935 +0000 UTC m=+22.690748232" watchObservedRunningTime="2026-09-19 09:06:49.004332129 +0000 UTC m=+22.691137425"
Sep 19 09:06:49.025676 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.025622    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jjlx7" podStartSLOduration=5.438971913 podStartE2EDuration="21.025600237s" totalImagesPullingTime="15.586628324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:49.025342296 +0000 UTC m=+22.712147602" watchObservedRunningTime="2026-09-19 09:06:49.025600237 +0000 UTC m=+22.712405534"
Sep 19 09:06:49.079595 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.079548    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-592rd" podStartSLOduration=1.551485118 podStartE2EDuration="21.079531766s" totalImagesPullingTime="19.528046648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:49.050089909 +0000 UTC m=+22.736895219" watchObservedRunningTime="2026-09-19 09:06:49.079531766 +0000 UTC m=+22.766337062"
Sep 19 09:06:49.079953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.079910    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4g6kn" podStartSLOduration=1.552112384 podStartE2EDuration="21.079901198s" totalImagesPullingTime="19.527788814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:49.079342864 +0000 UTC m=+22.766148164" watchObservedRunningTime="2026-09-19 09:06:49.079901198 +0000 UTC m=+22.766706497"
Sep 19 09:06:49.236821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.236786    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:49.237565 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.237544    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:49.259256 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.259208    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g6pzw" podStartSLOduration=1.6702982849999999 podStartE2EDuration="21.259190018s" totalImagesPullingTime="19.588891733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:49.102979781 +0000 UTC m=+22.789785079" watchObservedRunningTime="2026-09-19 09:06:49.259190018 +0000 UTC m=+22.945995316"
Sep 19 09:06:49.954655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.954628    2581 generic.go:356] "Generic (PLEG): container finished" podID="173d481c-5843-462c-b6fc-b645554ea972" containerID="5c710731fd8b0309d1ff6a5716227938f27e070ae5c3b7b53cd87cf039cc7e05" exitCode=0
Sep 19 09:06:49.955368 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.954715    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerDied","Data":"5c710731fd8b0309d1ff6a5716227938f27e070ae5c3b7b53cd87cf039cc7e05"}
Sep 19 09:06:49.956773 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.956747    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5zt8t" event={"ID":"d22aa59f-62ed-4687-a495-68cd7ba178aa","Type":"ContainerStarted","Data":"1ec0f39e7a3558175cbdca9076765af385c0969b1c3774f70e7ee697eac3b3f1"}
Sep 19 09:06:49.960061 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.960039    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"d7cb4af5d57ab0aaafd990037653e73d045a3e879d969c85030bde47130ed04e"}
Sep 19 09:06:49.960186 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.960136    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"fbd411b074b18b54c23456e2c64730247c98c72e90fb23deab77f24c8819b7a3"}
Sep 19 09:06:49.960186 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.960174    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"39c737fd4e19c737db862c34ed3b31af0f9302f5813977143cb8bb1e91d39413"}
Sep 19 09:06:49.960283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.960203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"d01a64d28a573d6695b5c725f026cbf313c0a98dda77313fb2f008170eb18f88"}
Sep 19 09:06:49.960826 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.960803    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:49.961345 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:49.961327    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-592rd"
Sep 19 09:06:50.034955 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.034912    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5zt8t" podStartSLOduration=2.454867308 podStartE2EDuration="22.034896121s" totalImagesPullingTime="19.580028813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:50.034664434 +0000 UTC m=+23.721469733" watchObservedRunningTime="2026-09-19 09:06:50.034896121 +0000 UTC m=+23.721701418"
Sep 19 09:06:50.062078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.062038    2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 09:06:50.808118 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.808012    2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T09:06:50.062057587Z","UUID":"7b15d7ae-1eff-4603-966c-190b1956a965","Handler":null,"Name":"","Endpoint":""}
Sep 19 09:06:50.809399 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.809380    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 09:06:50.809514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.809412    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 09:06:50.824808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.824778    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:50.824808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.824787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:50.824951 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.824787    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:50.824951 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:50.824889    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:50.825021 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:50.824948    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:50.825066 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:50.825022    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:50.965811 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:50.965716    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9" event={"ID":"f9f60cbc-401d-42fb-ad66-46b87ae25cc7","Type":"ContainerStarted","Data":"400cfffd9fe1036d6046c22460d925e9ecb0f584bb020530503bb6a197db1d4e"}
Sep 19 09:06:51.971497 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:51.971449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9" event={"ID":"f9f60cbc-401d-42fb-ad66-46b87ae25cc7","Type":"ContainerStarted","Data":"fb294f388e7c271b0377d7bd3cd3d9eea049603839d65885a0105a2e5062b129"}
Sep 19 09:06:51.975608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:51.975562    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"e204cd205941e30982ccf28dedda9f1463e73522a51db26efbce8c10126e7174"}
Sep 19 09:06:52.825503 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:52.825467    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:52.825503 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:52.825492    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:52.825743 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:52.825474    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:52.825743 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:52.825582    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:52.825743 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:52.825646    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:52.825892 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:52.825753    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:54.825052 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:54.824743    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:54.825052 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:54.824770    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:54.825052 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:54.825037    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:54.826541 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:54.824774    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:54.826541 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:54.825092    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:54.826541 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:54.825174    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:54.986648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:54.986479    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" event={"ID":"14038d22-868b-4a2f-beb9-b1a287b970d6","Type":"ContainerStarted","Data":"1127766d7a448d0638829b63eb4c326d20e08065a83e87db5744327f027a5ef4"}
Sep 19 09:06:54.986823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:54.986798    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:54.987670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:54.987648    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:55.003517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:55.003496    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:55.031594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:55.031551    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn" podStartSLOduration=7.480015504 podStartE2EDuration="27.031536761s" totalImagesPullingTime="19.551521257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:55.030615648 +0000 UTC m=+28.717420949" watchObservedRunningTime="2026-09-19 09:06:55.031536761 +0000 UTC m=+28.718342057"
Sep 19 09:06:55.031757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:55.031738    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8n5d9" podStartSLOduration=5.050278747 podStartE2EDuration="27.031733712s" totalImagesPullingTime="21.981454965s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:06:52.009456876 +0000 UTC m=+25.696262173" watchObservedRunningTime="2026-09-19 09:06:55.031733712 +0000 UTC m=+28.718539008"
Sep 19 09:06:55.992913 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:55.992878    2581 generic.go:356] "Generic (PLEG): container finished" podID="173d481c-5843-462c-b6fc-b645554ea972" containerID="9d6d9f8311d44b94ac0266e45ebfc435b02a38b03e9067c9e756226cacca64f7" exitCode=0
Sep 19 09:06:55.993712 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:55.992967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerDied","Data":"9d6d9f8311d44b94ac0266e45ebfc435b02a38b03e9067c9e756226cacca64f7"}
Sep 19 09:06:55.993712 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:55.993672    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:56.020336 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.020036    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:06:56.826139 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.825708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:56.826139 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:56.825793    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:56.826139 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.825844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:56.826139 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:56.825927    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:56.826139 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.825980    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:56.826139 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:56.826051    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:56.834686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.834660    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kr42v"]
Sep 19 09:06:56.834830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.834707    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nl6b5"]
Sep 19 09:06:56.834830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.834722    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-485pv"]
Sep 19 09:06:56.995661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.995630    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:56.995661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.995641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:56.996121 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:56.995637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:56.996121 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:56.995722    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:56.996121 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:56.995814    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:06:56.996241 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:56.996175    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:58.000005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:57.999974    2581 generic.go:356] "Generic (PLEG): container finished" podID="173d481c-5843-462c-b6fc-b645554ea972" containerID="b27481e5760abf34d7c444a0fffd460d27c3e31565e4b947c62c584fbe123e74" exitCode=0
Sep 19 09:06:58.000374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:58.000050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerDied","Data":"b27481e5760abf34d7c444a0fffd460d27c3e31565e4b947c62c584fbe123e74"}
Sep 19 09:06:58.825383 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:58.825324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:06:58.825593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:58.825325    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:06:58.825593 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:58.825448    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:06:58.825593 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:58.825562    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:06:58.825593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:06:58.825339    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:06:58.825783 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:06:58.825691    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:07:00.007138 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.007093    2581 generic.go:356] "Generic (PLEG): container finished" podID="173d481c-5843-462c-b6fc-b645554ea972" containerID="c76a87bd69881e509b8ed97ca908f140ce4a464ffbec7ec3bfd13abdf1c22e20" exitCode=0
Sep 19 09:07:00.007138 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.007130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerDied","Data":"c76a87bd69881e509b8ed97ca908f140ce4a464ffbec7ec3bfd13abdf1c22e20"}
Sep 19 09:07:00.496223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.496183    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:07:00.496402 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.496326    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 09:07:00.496402 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.496393    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret podName:60bb9f92-155a-4f69-b9d8-845edce54d11 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:32.496377891 +0000 UTC m=+66.183183166 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret") pod "global-pull-secret-syncer-kr42v" (UID: "60bb9f92-155a-4f69-b9d8-845edce54d11") : object "kube-system"/"original-pull-secret" not registered
Sep 19 09:07:00.596768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.596738    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:07:00.596947 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.596884    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:07:00.597000 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.596951    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:32.596935983 +0000 UTC m=+66.283741422 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 09:07:00.697875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.697840    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:07:00.698045 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.698011    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 09:07:00.698045 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.698039    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 09:07:00.698155 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.698052    2581 projected.go:196] Error preparing data for projected volume kube-api-access-w5mc2 for pod openshift-network-diagnostics/network-check-target-nl6b5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:07:00.698155 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.698133    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2 podName:3da542d9-bdb7-460e-a8dc-da326cf2cec0 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:32.69811687 +0000 UTC m=+66.384922148 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w5mc2" (UniqueName: "kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2") pod "network-check-target-nl6b5" (UID: "3da542d9-bdb7-460e-a8dc-da326cf2cec0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 09:07:00.824994 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.824905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:07:00.824994 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.824949    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:07:00.824994 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:00.824905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:07:00.825267 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.825036    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kr42v" podUID="60bb9f92-155a-4f69-b9d8-845edce54d11"
Sep 19 09:07:00.825267 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.825224    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-485pv" podUID="e0c101dc-58b0-481d-9859-234a79696788"
Sep 19 09:07:00.825378 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:00.825333    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nl6b5" podUID="3da542d9-bdb7-460e-a8dc-da326cf2cec0"
Sep 19 09:07:01.630194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.630011    2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-218.ec2.internal" event="NodeReady"
Sep 19 09:07:01.630667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.630322    2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 09:07:01.690801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.690769    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"]
Sep 19 09:07:01.694006 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.693983    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.697985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.697963    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 19 09:07:01.698766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.698719    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 19 09:07:01.698766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.698762    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2nnxz\""
Sep 19 09:07:01.698990 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.698771    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 19 09:07:01.703900 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.703858    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 19 09:07:01.714739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.714717    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-d7jp4"]
Sep 19 09:07:01.721437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.721413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.722880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.722840    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"]
Sep 19 09:07:01.724081 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.724046    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 09:07:01.725085 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.725070    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 09:07:01.729509 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.729458    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xchdv\""
Sep 19 09:07:01.732573 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.732550    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b5thm"]
Sep 19 09:07:01.736086 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.736070    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:01.744570 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.744551    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 09:07:01.744839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.744823    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 09:07:01.744902 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.744858    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5nz4g\""
Sep 19 09:07:01.752722 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.752699    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 09:07:01.752969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.752953    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d7jp4"]
Sep 19 09:07:01.754011 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.753693    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5thm"]
Sep 19 09:07:01.805861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.805825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.805870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-certificates\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.805901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0ad2ff9-8673-44cd-989f-893a35ed9c59-cert\") pod \"ingress-canary-b5thm\" (UID: \"c0ad2ff9-8673-44cd-989f-893a35ed9c59\") " pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:01.806062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.805945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsm9b\" (UniqueName: \"kubernetes.io/projected/c0ad2ff9-8673-44cd-989f-893a35ed9c59-kube-api-access-zsm9b\") pod \"ingress-canary-b5thm\" (UID: \"c0ad2ff9-8673-44cd-989f-893a35ed9c59\") " pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:01.806062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a06e1ca-d6f0-4504-bef6-b573918caa98-tmp-dir\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.806062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806060    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf8nc\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-kube-api-access-pf8nc\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806079    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzbs2\" (UniqueName: \"kubernetes.io/projected/0a06e1ca-d6f0-4504-bef6-b573918caa98-kube-api-access-vzbs2\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-trusted-ca\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-installation-pull-secrets\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7f45920-e15b-42e9-80bc-47f18cc1de7f-ca-trust-extracted\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-image-registry-private-configuration\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806217    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-bound-sa-token\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806236    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a06e1ca-d6f0-4504-bef6-b573918caa98-config-volume\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.806350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.806314    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.907321 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7f45920-e15b-42e9-80bc-47f18cc1de7f-ca-trust-extracted\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-image-registry-private-configuration\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-bound-sa-token\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a06e1ca-d6f0-4504-bef6-b573918caa98-config-volume\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.907530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907506    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.907726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-certificates\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0ad2ff9-8673-44cd-989f-893a35ed9c59-cert\") pod \"ingress-canary-b5thm\" (UID: \"c0ad2ff9-8673-44cd-989f-893a35ed9c59\") " pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:01.907726 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:01.907652    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 09:07:01.907726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsm9b\" (UniqueName: \"kubernetes.io/projected/c0ad2ff9-8673-44cd-989f-893a35ed9c59-kube-api-access-zsm9b\") pod \"ingress-canary-b5thm\" (UID: \"c0ad2ff9-8673-44cd-989f-893a35ed9c59\") " pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:01.907726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a06e1ca-d6f0-4504-bef6-b573918caa98-tmp-dir\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.907941 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907754    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7f45920-e15b-42e9-80bc-47f18cc1de7f-ca-trust-extracted\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907941 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:01.907775    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls podName:0a06e1ca-d6f0-4504-bef6-b573918caa98 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:02.407754943 +0000 UTC m=+36.094560219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls") pod "dns-default-d7jp4" (UID: "0a06e1ca-d6f0-4504-bef6-b573918caa98") : secret "dns-default-metrics-tls" not found
Sep 19 09:07:01.907941 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf8nc\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-kube-api-access-pf8nc\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907941 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzbs2\" (UniqueName: \"kubernetes.io/projected/0a06e1ca-d6f0-4504-bef6-b573918caa98-kube-api-access-vzbs2\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.907941 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-trusted-ca\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.907941 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-installation-pull-secrets\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.908175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907969    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a06e1ca-d6f0-4504-bef6-b573918caa98-tmp-dir\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.908175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.907971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a06e1ca-d6f0-4504-bef6-b573918caa98-config-volume\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.908175 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:01.908045    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 09:07:01.908175 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:01.908060    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57fcc9f7b7-ksdtf: secret "image-registry-tls" not found
Sep 19 09:07:01.908175 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:01.908121    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls podName:c7f45920-e15b-42e9-80bc-47f18cc1de7f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:02.408094518 +0000 UTC m=+36.094899793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls") pod "image-registry-57fcc9f7b7-ksdtf" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f") : secret "image-registry-tls" not found
Sep 19 09:07:01.908717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.908668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-certificates\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.909049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.909021    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-trusted-ca\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.912571 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.912548    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-image-registry-private-configuration\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.912695 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.912553    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-installation-pull-secrets\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.912695 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.912616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0ad2ff9-8673-44cd-989f-893a35ed9c59-cert\") pod \"ingress-canary-b5thm\" (UID: \"c0ad2ff9-8673-44cd-989f-893a35ed9c59\") " pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:01.927881 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.927854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf8nc\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-kube-api-access-pf8nc\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:01.928030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.927854    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsm9b\" (UniqueName: \"kubernetes.io/projected/c0ad2ff9-8673-44cd-989f-893a35ed9c59-kube-api-access-zsm9b\") pod \"ingress-canary-b5thm\" (UID: \"c0ad2ff9-8673-44cd-989f-893a35ed9c59\") " pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:01.929989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.929940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzbs2\" (UniqueName: \"kubernetes.io/projected/0a06e1ca-d6f0-4504-bef6-b573918caa98-kube-api-access-vzbs2\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:01.932738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:01.932718    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-bound-sa-token\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:02.044624 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.044585    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b5thm"
Sep 19 09:07:02.353296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.353143    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b5thm"]
Sep 19 09:07:02.357043 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:02.357013    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ad2ff9_8673_44cd_989f_893a35ed9c59.slice/crio-4065810c7ef3e0026f63e844cd22659ddab750878e32d8752ece37f5cd81df3c WatchSource:0}: Error finding container 4065810c7ef3e0026f63e844cd22659ddab750878e32d8752ece37f5cd81df3c: Status 404 returned error can't find the container with id 4065810c7ef3e0026f63e844cd22659ddab750878e32d8752ece37f5cd81df3c
Sep 19 09:07:02.412674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.412636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:02.412870 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.412687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:02.412870 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:02.412783    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 09:07:02.412870 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:02.412794    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57fcc9f7b7-ksdtf: secret "image-registry-tls" not found
Sep 19 09:07:02.412870 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:02.412798    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 09:07:02.412870 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:02.412844    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls podName:c7f45920-e15b-42e9-80bc-47f18cc1de7f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:03.412831027 +0000 UTC m=+37.099636303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls") pod "image-registry-57fcc9f7b7-ksdtf" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f") : secret "image-registry-tls" not found
Sep 19 09:07:02.413056 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:02.412918    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls podName:0a06e1ca-d6f0-4504-bef6-b573918caa98 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:03.412897649 +0000 UTC m=+37.099702925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls") pod "dns-default-d7jp4" (UID: "0a06e1ca-d6f0-4504-bef6-b573918caa98") : secret "dns-default-metrics-tls" not found
Sep 19 09:07:02.825604 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.825572    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:07:02.827737 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.826078    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:07:02.827737 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.826257    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:07:02.828405 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.828354    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 09:07:02.829823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.829802    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fszt9\""
Sep 19 09:07:02.829966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.829951    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 09:07:02.830171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.830152    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 09:07:02.830227 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.830154    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 09:07:02.830327 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:02.830156    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2r86\""
Sep 19 09:07:03.017021 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:03.016928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5thm" event={"ID":"c0ad2ff9-8673-44cd-989f-893a35ed9c59","Type":"ContainerStarted","Data":"4065810c7ef3e0026f63e844cd22659ddab750878e32d8752ece37f5cd81df3c"}
Sep 19 09:07:03.421910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:03.421204    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:03.421910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:03.421277    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:03.421910 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:03.421434    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 09:07:03.421910 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:03.421449    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57fcc9f7b7-ksdtf: secret "image-registry-tls" not found
Sep 19 09:07:03.421910 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:03.421504    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls podName:c7f45920-e15b-42e9-80bc-47f18cc1de7f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:05.421486793 +0000 UTC m=+39.108292072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls") pod "image-registry-57fcc9f7b7-ksdtf" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f") : secret "image-registry-tls" not found
Sep 19 09:07:03.421910 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:03.421570    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 09:07:03.421910 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:03.421604    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls podName:0a06e1ca-d6f0-4504-bef6-b573918caa98 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:05.421592991 +0000 UTC m=+39.108398266 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls") pod "dns-default-d7jp4" (UID: "0a06e1ca-d6f0-4504-bef6-b573918caa98") : secret "dns-default-metrics-tls" not found
Sep 19 09:07:05.438190 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:05.437977    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:05.438759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:05.438142    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 09:07:05.438759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:05.438284    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57fcc9f7b7-ksdtf: secret "image-registry-tls" not found
Sep 19 09:07:05.438759 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:05.438314    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:05.438759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:05.438361    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls podName:c7f45920-e15b-42e9-80bc-47f18cc1de7f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:09.438338488 +0000 UTC m=+43.125143782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls") pod "image-registry-57fcc9f7b7-ksdtf" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f") : secret "image-registry-tls" not found
Sep 19 09:07:05.438759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:05.438465    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 09:07:05.438759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:05.438524    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls podName:0a06e1ca-d6f0-4504-bef6-b573918caa98 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:09.438508086 +0000 UTC m=+43.125313375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls") pod "dns-default-d7jp4" (UID: "0a06e1ca-d6f0-4504-bef6-b573918caa98") : secret "dns-default-metrics-tls" not found
Sep 19 09:07:05.964184 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:05.964080    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:06.026931 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:06.026897    2581 generic.go:356] "Generic (PLEG): container finished" podID="173d481c-5843-462c-b6fc-b645554ea972" containerID="357bf20e1f4bd9a591ef78b020e87d541764bc3b921a184a50e1e28a5f6e6f0a" exitCode=0
Sep 19 09:07:06.027153 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:06.026978    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerDied","Data":"357bf20e1f4bd9a591ef78b020e87d541764bc3b921a184a50e1e28a5f6e6f0a"}
Sep 19 09:07:06.028523 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:06.028501    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b5thm" event={"ID":"c0ad2ff9-8673-44cd-989f-893a35ed9c59","Type":"ContainerStarted","Data":"ff55e78eeb1953ad7d5a9c151bc1d63eb750c6ac0302f104a516439ded311c98"}
Sep 19 09:07:06.098039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:06.097990    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b5thm" podStartSLOduration=1.979868586 podStartE2EDuration="5.097968957s" totalImagesPullingTime="3.118100371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:06.09776237 +0000 UTC m=+39.784567666" watchObservedRunningTime="2026-09-19 09:07:06.097968957 +0000 UTC m=+39.784774255"
Sep 19 09:07:06.128211 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:06.126522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:07.033822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:07.033787    2581 generic.go:356] "Generic (PLEG): container finished" podID="173d481c-5843-462c-b6fc-b645554ea972" containerID="0f62f907d474e676758a60baa0598711092f6a16a7ea49e9fdadc6ed05052766" exitCode=0
Sep 19 09:07:07.034279 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:07.033876    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerDied","Data":"0f62f907d474e676758a60baa0598711092f6a16a7ea49e9fdadc6ed05052766"}
Sep 19 09:07:08.039642 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:08.039611    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" event={"ID":"173d481c-5843-462c-b6fc-b645554ea972","Type":"ContainerStarted","Data":"6a17522091e5b8e59b2127de55bdd2b154f4f7d3354f64f1336d4a0e2d7f358e"}
Sep 19 09:07:08.293573 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:08.293499    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:08.536991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:08.536962    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:09.473326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:09.473289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:09.473783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:09.473380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:09.473783 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:09.473457    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 09:07:09.473783 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:09.473460    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 09:07:09.473783 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:09.473472    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57fcc9f7b7-ksdtf: secret "image-registry-tls" not found
Sep 19 09:07:09.473783 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:09.473524    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls podName:0a06e1ca-d6f0-4504-bef6-b573918caa98 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:17.473511736 +0000 UTC m=+51.160317011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls") pod "dns-default-d7jp4" (UID: "0a06e1ca-d6f0-4504-bef6-b573918caa98") : secret "dns-default-metrics-tls" not found
Sep 19 09:07:09.473783 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:09.473539    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls podName:c7f45920-e15b-42e9-80bc-47f18cc1de7f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:17.473533173 +0000 UTC m=+51.160338449 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls") pod "image-registry-57fcc9f7b7-ksdtf" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f") : secret "image-registry-tls" not found
Sep 19 09:07:12.161143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:12.161089    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:13.205073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:13.205023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:13.334790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:13.334738    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:13.977495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:13.977461    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:14.084645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:14.084618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:16.381088 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:16.381057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:16.497264 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:16.497235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:17.533308 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.533272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:17.533308 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.533322    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:17.533759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:17.533429    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 19 09:07:17.533759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:17.533429    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 09:07:17.533759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:17.533511    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls podName:0a06e1ca-d6f0-4504-bef6-b573918caa98 nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.533494593 +0000 UTC m=+67.220299868 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls") pod "dns-default-d7jp4" (UID: "0a06e1ca-d6f0-4504-bef6-b573918caa98") : secret "dns-default-metrics-tls" not found
Sep 19 09:07:17.533759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:17.533439    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57fcc9f7b7-ksdtf: secret "image-registry-tls" not found
Sep 19 09:07:17.533759 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:17.533576    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls podName:c7f45920-e15b-42e9-80bc-47f18cc1de7f nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.533565148 +0000 UTC m=+67.220370423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls") pod "image-registry-57fcc9f7b7-ksdtf" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f") : secret "image-registry-tls" not found
Sep 19 09:07:17.816735 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.816636    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hc2pd" podStartSLOduration=17.422196789 podStartE2EDuration="49.816619277s" totalImagesPullingTime="32.394422488s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:08.077054506 +0000 UTC m=+41.763859804" watchObservedRunningTime="2026-09-19 09:07:17.816619277 +0000 UTC m=+51.503424573"
Sep 19 09:07:17.817167 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.817152    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"]
Sep 19 09:07:17.840515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.840489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"]
Sep 19 09:07:17.840669 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.840588    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:17.844685 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.844664    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-96lgx\""
Sep 19 09:07:17.844853 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.844838    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 09:07:17.847501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.847481    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 19 09:07:17.847622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.847483    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 19 09:07:17.854641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.854625    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 09:07:17.868866 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.868844    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"]
Sep 19 09:07:17.890539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.890517    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"
Sep 19 09:07:17.895779 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.895755    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 19 09:07:17.896782 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.896758    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 19 09:07:17.897028 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.897012    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rq2t4\""
Sep 19 09:07:17.900707 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.900688    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"]
Sep 19 09:07:17.927170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.927139    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-v684g"]
Sep 19 09:07:17.936537 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.936513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:17.936652 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.936577    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:17.936652 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.936595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmpq8\" (UniqueName: \"kubernetes.io/projected/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-kube-api-access-wmpq8\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:17.955024 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.955001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:17.959570 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.959492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 19 09:07:17.959861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.959833    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 19 09:07:17.960336 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.960316    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-v684g"]
Sep 19 09:07:17.967481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.967460    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 09:07:17.967591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.967500    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 09:07:17.971896 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.971878    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dhbqh\""
Sep 19 09:07:17.972761 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:17.972745    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 19 09:07:18.033998 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.033966    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"]
Sep 19 09:07:18.037342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.037319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:18.037434 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.037419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:18.037477 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.037439    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmpq8\" (UniqueName: \"kubernetes.io/projected/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-kube-api-access-wmpq8\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:18.037477 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.037458    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx5nw\" (UniqueName: \"kubernetes.io/projected/cd8bb87f-3e17-4520-924c-6f60156337fc-kube-api-access-hx5nw\") pod \"volume-data-source-validator-5c4db8bdf9-jtvdj\" (UID: \"cd8bb87f-3e17-4520-924c-6f60156337fc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"
Sep 19 09:07:18.037603 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.037585    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:18.037680 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.037669    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls podName:6e7c3e3d-26d5-4268-88fa-70e24f64d7ed nodeName:}" failed. No retries permitted until 2026-09-19 09:07:18.53765045 +0000 UTC m=+52.224455742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-98ptl" (UID: "6e7c3e3d-26d5-4268-88fa-70e24f64d7ed") : secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:18.037934 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.037915    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:18.052073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.052054    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"]
Sep 19 09:07:18.052207 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.052189    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.056278 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.056254    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmpq8\" (UniqueName: \"kubernetes.io/projected/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-kube-api-access-wmpq8\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:18.056421 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.056407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 19 09:07:18.056513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.056495    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 19 09:07:18.056635 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.056502    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 19 09:07:18.056814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.056798    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-z8bgp\""
Sep 19 09:07:18.057012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.056995    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 19 09:07:18.068273 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.068222    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:18.073177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.073155    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 19 09:07:18.073672 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.073652    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 19 09:07:18.073769 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.073717    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qth4x\""
Sep 19 09:07:18.080765 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.080743    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"]
Sep 19 09:07:18.083793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.083771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"]
Sep 19 09:07:18.138823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.138791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx5nw\" (UniqueName: \"kubernetes.io/projected/cd8bb87f-3e17-4520-924c-6f60156337fc-kube-api-access-hx5nw\") pod \"volume-data-source-validator-5c4db8bdf9-jtvdj\" (UID: \"cd8bb87f-3e17-4520-924c-6f60156337fc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"
Sep 19 09:07:18.139001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.138865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4962a821-49df-4abb-ae80-375b5d2724fd-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.139001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.138890    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4962a821-49df-4abb-ae80-375b5d2724fd-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.139001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.138922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4962a821-49df-4abb-ae80-375b5d2724fd-tmp\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.139150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.139002    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr2ts\" (UniqueName: \"kubernetes.io/projected/4962a821-49df-4abb-ae80-375b5d2724fd-kube-api-access-kr2ts\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.139150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.139037    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4962a821-49df-4abb-ae80-375b5d2724fd-snapshots\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.139150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.139069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4962a821-49df-4abb-ae80-375b5d2724fd-serving-cert\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.154158 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.154123    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx5nw\" (UniqueName: \"kubernetes.io/projected/cd8bb87f-3e17-4520-924c-6f60156337fc-kube-api-access-hx5nw\") pod \"volume-data-source-validator-5c4db8bdf9-jtvdj\" (UID: \"cd8bb87f-3e17-4520-924c-6f60156337fc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"
Sep 19 09:07:18.199538 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.199501    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"
Sep 19 09:07:18.240421 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4962a821-49df-4abb-ae80-375b5d2724fd-serving-cert\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.240539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240432    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bc99bcb-8487-4ae8-89e6-de3272b082f1-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.240539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:18.240619 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4962a821-49df-4abb-ae80-375b5d2724fd-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.240619 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240601    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4962a821-49df-4abb-ae80-375b5d2724fd-snapshots\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.240691 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmjmz\" (UniqueName: \"kubernetes.io/projected/3bc99bcb-8487-4ae8-89e6-de3272b082f1-kube-api-access-pmjmz\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.240691 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4962a821-49df-4abb-ae80-375b5d2724fd-tmp\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.240772 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:18.240772 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4962a821-49df-4abb-ae80-375b5d2724fd-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.240867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240791    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bc99bcb-8487-4ae8-89e6-de3272b082f1-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.240867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.240819    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr2ts\" (UniqueName: \"kubernetes.io/projected/4962a821-49df-4abb-ae80-375b5d2724fd-kube-api-access-kr2ts\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.241185 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.241164    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4962a821-49df-4abb-ae80-375b5d2724fd-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.241283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.241260    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4962a821-49df-4abb-ae80-375b5d2724fd-tmp\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.241322 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.241304    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/4962a821-49df-4abb-ae80-375b5d2724fd-snapshots\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.241618 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.241599    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4962a821-49df-4abb-ae80-375b5d2724fd-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.242908 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.242888    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4962a821-49df-4abb-ae80-375b5d2724fd-serving-cert\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.255999 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.255975    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr2ts\" (UniqueName: \"kubernetes.io/projected/4962a821-49df-4abb-ae80-375b5d2724fd-kube-api-access-kr2ts\") pod \"insights-operator-6f9c644dd9-v684g\" (UID: \"4962a821-49df-4abb-ae80-375b5d2724fd\") " pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.264454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.264430    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-v684g"
Sep 19 09:07:18.287839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.287722    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:18.341725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.341691    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bc99bcb-8487-4ae8-89e6-de3272b082f1-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.341830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.341743    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:18.341830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.341782    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmjmz\" (UniqueName: \"kubernetes.io/projected/3bc99bcb-8487-4ae8-89e6-de3272b082f1-kube-api-access-pmjmz\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.341830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.341812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:18.341923 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.341886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bc99bcb-8487-4ae8-89e6-de3272b082f1-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.341959 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.341915    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 19 09:07:18.342027 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.341983    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert podName:a2870d9c-99a3-4533-a2d7-9fe87b230b6b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:18.841964444 +0000 UTC m=+52.528769719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-k95hw" (UID: "a2870d9c-99a3-4533-a2d7-9fe87b230b6b") : secret "networking-console-plugin-cert" not found
Sep 19 09:07:18.342401 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.342381    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3bc99bcb-8487-4ae8-89e6-de3272b082f1-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.342535 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.342519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:18.344160 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.344145    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3bc99bcb-8487-4ae8-89e6-de3272b082f1-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.364588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.364555    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmjmz\" (UniqueName: \"kubernetes.io/projected/3bc99bcb-8487-4ae8-89e6-de3272b082f1-kube-api-access-pmjmz\") pod \"kube-storage-version-migrator-operator-55889f7b47-p2m8b\" (UID: \"3bc99bcb-8487-4ae8-89e6-de3272b082f1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.365443 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.365429    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"
Sep 19 09:07:18.440596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.440527    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:18.519912 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.519880    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj"]
Sep 19 09:07:18.523740 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:18.523700    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8bb87f_3e17_4520_924c_6f60156337fc.slice/crio-96b7ecdb85d8f6067c5801efba8505209c9623313f26f5f0b32668e2dd534842 WatchSource:0}: Error finding container 96b7ecdb85d8f6067c5801efba8505209c9623313f26f5f0b32668e2dd534842: Status 404 returned error can't find the container with id 96b7ecdb85d8f6067c5801efba8505209c9623313f26f5f0b32668e2dd534842
Sep 19 09:07:18.543830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.543805    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:18.544173 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.543956    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:18.544173 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.544025    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls podName:6e7c3e3d-26d5-4268-88fa-70e24f64d7ed nodeName:}" failed. No retries permitted until 2026-09-19 09:07:19.544009486 +0000 UTC m=+53.230814761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-98ptl" (UID: "6e7c3e3d-26d5-4268-88fa-70e24f64d7ed") : secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:18.584810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.584734    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-v684g"]
Sep 19 09:07:18.587642 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:18.587613    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4962a821_49df_4abb_ae80_375b5d2724fd.slice/crio-9fdce4bb37d2c0797b2fbdc01a957227887362021f93ef80d92a3e0b616f97c6 WatchSource:0}: Error finding container 9fdce4bb37d2c0797b2fbdc01a957227887362021f93ef80d92a3e0b616f97c6: Status 404 returned error can't find the container with id 9fdce4bb37d2c0797b2fbdc01a957227887362021f93ef80d92a3e0b616f97c6
Sep 19 09:07:18.641678 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.641644    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:18.672779 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.672750    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b"]
Sep 19 09:07:18.676316 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:18.676292    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc99bcb_8487_4ae8_89e6_de3272b082f1.slice/crio-63939c8359548bec6ad94282166c4c8f9d9a9eff51c19da7e2ecd3a046b164c6 WatchSource:0}: Error finding container 63939c8359548bec6ad94282166c4c8f9d9a9eff51c19da7e2ecd3a046b164c6: Status 404 returned error can't find the container with id 63939c8359548bec6ad94282166c4c8f9d9a9eff51c19da7e2ecd3a046b164c6
Sep 19 09:07:18.759488 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.759454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:18.846777 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.846704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:18.847035 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.846815    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 19 09:07:18.847035 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:18.846874    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert podName:a2870d9c-99a3-4533-a2d7-9fe87b230b6b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:19.846856473 +0000 UTC m=+53.533661749 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-k95hw" (UID: "a2870d9c-99a3-4533-a2d7-9fe87b230b6b") : secret "networking-console-plugin-cert" not found
Sep 19 09:07:18.867719 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:18.867688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:19.067915 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:19.067879    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj" event={"ID":"cd8bb87f-3e17-4520-924c-6f60156337fc","Type":"ContainerStarted","Data":"96b7ecdb85d8f6067c5801efba8505209c9623313f26f5f0b32668e2dd534842"}
Sep 19 09:07:19.069221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:19.069193    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b" event={"ID":"3bc99bcb-8487-4ae8-89e6-de3272b082f1","Type":"ContainerStarted","Data":"63939c8359548bec6ad94282166c4c8f9d9a9eff51c19da7e2ecd3a046b164c6"}
Sep 19 09:07:19.070209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:19.070188    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-v684g" event={"ID":"4962a821-49df-4abb-ae80-375b5d2724fd","Type":"ContainerStarted","Data":"9fdce4bb37d2c0797b2fbdc01a957227887362021f93ef80d92a3e0b616f97c6"}
Sep 19 09:07:19.554620 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:19.554576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:19.555133 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:19.554757    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:19.555133 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:19.554830    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls podName:6e7c3e3d-26d5-4268-88fa-70e24f64d7ed nodeName:}" failed. No retries permitted until 2026-09-19 09:07:21.554812837 +0000 UTC m=+55.241618112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-98ptl" (UID: "6e7c3e3d-26d5-4268-88fa-70e24f64d7ed") : secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:19.857808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:19.857716    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:19.857981 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:19.857883    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 19 09:07:19.857981 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:19.857957    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert podName:a2870d9c-99a3-4533-a2d7-9fe87b230b6b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:21.85793762 +0000 UTC m=+55.544742907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-k95hw" (UID: "a2870d9c-99a3-4533-a2d7-9fe87b230b6b") : secret "networking-console-plugin-cert" not found
Sep 19 09:07:20.327699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:20.327666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:20.470488 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:20.470456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:21.251406 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:21.251372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:21.425154 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:21.425125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:21.572514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:21.572418    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:21.572680 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:21.572595    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:21.572745 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:21.572680    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls podName:6e7c3e3d-26d5-4268-88fa-70e24f64d7ed nodeName:}" failed. No retries permitted until 2026-09-19 09:07:25.57265663 +0000 UTC m=+59.259461923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-98ptl" (UID: "6e7c3e3d-26d5-4268-88fa-70e24f64d7ed") : secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:21.875825 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:21.875723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:21.875995 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:21.875880    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 19 09:07:21.875995 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:21.875962    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert podName:a2870d9c-99a3-4533-a2d7-9fe87b230b6b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:25.87594041 +0000 UTC m=+59.562745689 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-k95hw" (UID: "a2870d9c-99a3-4533-a2d7-9fe87b230b6b") : secret "networking-console-plugin-cert" not found
Sep 19 09:07:22.079320 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.079283    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-v684g" event={"ID":"4962a821-49df-4abb-ae80-375b5d2724fd","Type":"ContainerStarted","Data":"2e7a892ec214000d8ad5cd5a495d5b75fff2dade88446fb10c9887bad2bf098d"}
Sep 19 09:07:22.085485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.085445    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj" event={"ID":"cd8bb87f-3e17-4520-924c-6f60156337fc","Type":"ContainerStarted","Data":"71be9fcf39554a099122bd46f8846fdeb2df4426355d387153426c75abea75d5"}
Sep 19 09:07:22.087080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.087056    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b" event={"ID":"3bc99bcb-8487-4ae8-89e6-de3272b082f1","Type":"ContainerStarted","Data":"9a9d36df1f721d78c84a9a0995e40deb9429162349bf982ca6b637c7c9d3474a"}
Sep 19 09:07:22.119570 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.119501    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-v684g" podStartSLOduration=2.4166833 podStartE2EDuration="5.119485675s" totalImagesPullingTime="2.702802375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:22.119450975 +0000 UTC m=+55.806256267" watchObservedRunningTime="2026-09-19 09:07:22.119485675 +0000 UTC m=+55.806290966"
Sep 19 09:07:22.215941 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.215878    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-jtvdj" podStartSLOduration=2.4544525249999998 podStartE2EDuration="5.215857788s" totalImagesPullingTime="2.761405263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:22.215160338 +0000 UTC m=+55.901965635" watchObservedRunningTime="2026-09-19 09:07:22.215857788 +0000 UTC m=+55.902663085"
Sep 19 09:07:22.216163 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.216050    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b" podStartSLOduration=2.596960627 podStartE2EDuration="5.216042938s" totalImagesPullingTime="2.619082311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:22.155383843 +0000 UTC m=+55.842189142" watchObservedRunningTime="2026-09-19 09:07:22.216042938 +0000 UTC m=+55.902848237"
Sep 19 09:07:22.500598 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.500512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:22.658042 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.658005    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"]
Sep 19 09:07:22.661669 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.661641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"
Sep 19 09:07:22.664561 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.664537    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 19 09:07:22.665286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.665265    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 19 09:07:22.668271 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.668253    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-7l9dr\""
Sep 19 09:07:22.680987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.680966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:22.682091 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.682068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbhpt\" (UniqueName: \"kubernetes.io/projected/35e70a78-17a8-4f84-9e76-561b60cbb75e-kube-api-access-hbhpt\") pod \"migrator-76794cbd6c-ms6vs\" (UID: \"35e70a78-17a8-4f84-9e76-561b60cbb75e\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"
Sep 19 09:07:22.699512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.699489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"]
Sep 19 09:07:22.782952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.782862    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbhpt\" (UniqueName: \"kubernetes.io/projected/35e70a78-17a8-4f84-9e76-561b60cbb75e-kube-api-access-hbhpt\") pod \"migrator-76794cbd6c-ms6vs\" (UID: \"35e70a78-17a8-4f84-9e76-561b60cbb75e\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"
Sep 19 09:07:22.827554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.827516    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbhpt\" (UniqueName: \"kubernetes.io/projected/35e70a78-17a8-4f84-9e76-561b60cbb75e-kube-api-access-hbhpt\") pod \"migrator-76794cbd6c-ms6vs\" (UID: \"35e70a78-17a8-4f84-9e76-561b60cbb75e\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"
Sep 19 09:07:22.917857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.917823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:22.953217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.953186    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7m7zt"]
Sep 19 09:07:22.956304 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.956283    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:22.961860 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.961842    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 09:07:22.962122 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.962088    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 09:07:22.962237 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.962218    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w276g\""
Sep 19 09:07:22.972267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.972249    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"
Sep 19 09:07:22.979271 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.979250    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7m7zt"]
Sep 19 09:07:22.985448 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.985421    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c83865a1-9ca7-46ec-9edf-75a9899f261a-crio-socket\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:22.985554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.985523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:22.985617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.985558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c83865a1-9ca7-46ec-9edf-75a9899f261a-data-volume\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:22.985617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.985589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wczj7\" (UniqueName: \"kubernetes.io/projected/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-api-access-wczj7\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:22.985617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:22.985613    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.086639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.086553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c83865a1-9ca7-46ec-9edf-75a9899f261a-crio-socket\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.086794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.086636    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.086794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.086669    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c83865a1-9ca7-46ec-9edf-75a9899f261a-data-volume\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.086794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.086698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wczj7\" (UniqueName: \"kubernetes.io/projected/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-api-access-wczj7\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.086794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.086707    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c83865a1-9ca7-46ec-9edf-75a9899f261a-crio-socket\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.086794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.086726    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.087033 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.086808    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:23.087033 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.086848    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 09:07:23.087033 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.086870    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:23.586852646 +0000 UTC m=+57.273657927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:23.087033 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.086922    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:23.586904269 +0000 UTC m=+57.273709569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : configmap "kube-rbac-proxy" not found
Sep 19 09:07:23.087227 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.087190    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c83865a1-9ca7-46ec-9edf-75a9899f261a-data-volume\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.087571 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.087544    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:23.108884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.108851    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wczj7\" (UniqueName: \"kubernetes.io/projected/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-api-access-wczj7\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.256878 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.256856    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs"]
Sep 19 09:07:23.258143 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:23.258121    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e70a78_17a8_4f84_9e76_561b60cbb75e.slice/crio-71f0673ef30713d8f8a79dd7feeb1279e9fe806d4c485e17d82766537a0c1b61 WatchSource:0}: Error finding container 71f0673ef30713d8f8a79dd7feeb1279e9fe806d4c485e17d82766537a0c1b61: Status 404 returned error can't find the container with id 71f0673ef30713d8f8a79dd7feeb1279e9fe806d4c485e17d82766537a0c1b61
Sep 19 09:07:23.591524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.591493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.591941 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:23.591556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:23.591941 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.591649    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:23.591941 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.591674    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 09:07:23.591941 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.591729    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:24.591708054 +0000 UTC m=+58.278513330 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:23.591941 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:23.591743    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:24.591736447 +0000 UTC m=+58.278541722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : configmap "kube-rbac-proxy" not found
Sep 19 09:07:24.093041 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:24.093008    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs" event={"ID":"35e70a78-17a8-4f84-9e76-561b60cbb75e","Type":"ContainerStarted","Data":"71f0673ef30713d8f8a79dd7feeb1279e9fe806d4c485e17d82766537a0c1b61"}
Sep 19 09:07:24.550549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:24.550522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:24.599735 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:24.599706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:24.600076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:24.599761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:24.600076 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:24.599829    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:24.600076 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:24.599851    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 09:07:24.600076 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:24.599887    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:26.599872644 +0000 UTC m=+60.286677919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:24.600076 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:24.599913    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:26.599897499 +0000 UTC m=+60.286702793 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : configmap "kube-rbac-proxy" not found
Sep 19 09:07:24.733462 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:24.733369    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:24.899328 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:24.899285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:24.983333 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:24.983308    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4g6kn_c80ec8c6-a38b-496c-baaa-33d918287237/dns-node-resolver/0.log"
Sep 19 09:07:25.098025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.097951    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs" event={"ID":"35e70a78-17a8-4f84-9e76-561b60cbb75e","Type":"ContainerStarted","Data":"835c1fd755165dcb78c0ff8314f2ecda4d8c77b3b823062f2950f359656a52f6"}
Sep 19 09:07:25.098025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.098000    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs" event={"ID":"35e70a78-17a8-4f84-9e76-561b60cbb75e","Type":"ContainerStarted","Data":"80a3cf9c9c4e4071655a1ed04bc41238c7e6ce590e81602aab35ba65bc37fca0"}
Sep 19 09:07:25.214306 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.214249    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-ms6vs" podStartSLOduration=2.001570366 podStartE2EDuration="3.214231803s" totalImagesPullingTime="1.212661437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:25.213315507 +0000 UTC m=+58.900120803" watchObservedRunningTime="2026-09-19 09:07:25.214231803 +0000 UTC m=+58.901037097"
Sep 19 09:07:25.316305 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.316269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:25.531384 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.531349    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:25.609071 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.609036    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:25.609470 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:25.609208    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:25.609470 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:25.609283    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls podName:6e7c3e3d-26d5-4268-88fa-70e24f64d7ed nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.60926543 +0000 UTC m=+67.296070709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-98ptl" (UID: "6e7c3e3d-26d5-4268-88fa-70e24f64d7ed") : secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:25.668398 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.668356    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:25.911042 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.911010    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:25.911214 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:25.911189    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 19 09:07:25.911273 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:25.911262    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert podName:a2870d9c-99a3-4533-a2d7-9fe87b230b6b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:33.911243209 +0000 UTC m=+67.598048502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-k95hw" (UID: "a2870d9c-99a3-4533-a2d7-9fe87b230b6b") : secret "networking-console-plugin-cert" not found
Sep 19 09:07:25.988894 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:25.988865    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jjlx7_10fdafc3-1641-4fa0-8ec1-3d403c30e9b5/node-ca/0.log"
Sep 19 09:07:26.617209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:26.617173    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:26.617565 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:26.617215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:26.617565 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:26.617310    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 09:07:26.617565 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:26.617324    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:26.617565 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:26.617383    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:30.617364703 +0000 UTC m=+64.304169978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:26.617565 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:26.617397    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:30.617391037 +0000 UTC m=+64.304196312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : configmap "kube-rbac-proxy" not found
Sep 19 09:07:26.990410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:26.990384    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5thm_c0ad2ff9-8673-44cd-989f-893a35ed9c59/serve-healthcheck-canary/0.log"
Sep 19 09:07:26.990608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:26.990552    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:27.189575 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:27.189544    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-ms6vs_35e70a78-17a8-4f84-9e76-561b60cbb75e/migrator/0.log"
Sep 19 09:07:27.262057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:27.261979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:27.381346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:27.381317    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-ms6vs_35e70a78-17a8-4f84-9e76-561b60cbb75e/graceful-termination/0.log"
Sep 19 09:07:27.578131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:27.578023    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-p2m8b_3bc99bcb-8487-4ae8-89e6-de3272b082f1/kube-storage-version-migrator-operator/0.log"
Sep 19 09:07:28.016525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:28.016497    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vk7xn"
Sep 19 09:07:28.500834 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:28.500799    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:28.777930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:28.777841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:28.917810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:28.917776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:29.130183 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:29.130154    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:29.367723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:29.367689    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:29.535755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:29.535676    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:30.183296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:30.183259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:30.566750 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:30.566670    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:30.647174 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:30.647142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:30.647285 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:30.647267    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 09:07:30.647340 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:30.647328    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:38.647307573 +0000 UTC m=+72.334112848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : configmap "kube-rbac-proxy" not found
Sep 19 09:07:30.647397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:30.647271    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:30.647397 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:30.647351    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:30.647489 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:30.647453    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls podName:c83865a1-9ca7-46ec-9edf-75a9899f261a nodeName:}" failed. No retries permitted until 2026-09-19 09:07:38.647432553 +0000 UTC m=+72.334237846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7m7zt" (UID: "c83865a1-9ca7-46ec-9edf-75a9899f261a") : secret "insights-runtime-extractor-tls" not found
Sep 19 09:07:31.033393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:31.033362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:31.373457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:31.373374    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:31.823151 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:31.823115    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:32.030763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.030716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:32.376182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.376145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:32.563718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.563679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:07:32.566225 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.566210    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 09:07:32.576781 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.576764    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/60bb9f92-155a-4f69-b9d8-845edce54d11-original-pull-secret\") pod \"global-pull-secret-syncer-kr42v\" (UID: \"60bb9f92-155a-4f69-b9d8-845edce54d11\") " pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:07:32.632920 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.632821    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:32.664629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.664599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:07:32.667280 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.667260    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 09:07:32.675171 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:32.675148    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 19 09:07:32.675240 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:32.675209    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs podName:e0c101dc-58b0-481d-9859-234a79696788 nodeName:}" failed. No retries permitted until 2026-09-19 09:08:36.675191553 +0000 UTC m=+130.361996829 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs") pod "network-metrics-daemon-485pv" (UID: "e0c101dc-58b0-481d-9859-234a79696788") : secret "metrics-daemon-secret" not found
Sep 19 09:07:32.765888 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.765851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:07:32.784926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.784895    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 09:07:32.790491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.790475    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 09:07:32.799562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.799523    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5mc2\" (UniqueName: \"kubernetes.io/projected/3da542d9-bdb7-460e-a8dc-da326cf2cec0-kube-api-access-w5mc2\") pod \"network-check-target-nl6b5\" (UID: \"3da542d9-bdb7-460e-a8dc-da326cf2cec0\") " pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:07:32.839801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.839771    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kr42v"
Sep 19 09:07:32.858671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.858648    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2r86\""
Sep 19 09:07:32.865510 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.865473    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:07:32.881632 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:32.881603    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:33.128810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.128790    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kr42v"]
Sep 19 09:07:33.131491 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:33.131462    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bb9f92_155a_4f69_b9d8_845edce54d11.slice/crio-dc0f0709b14a6732b97ce6a0d1f953e1eca19855355c2ef57b599c98a7e8fcde WatchSource:0}: Error finding container dc0f0709b14a6732b97ce6a0d1f953e1eca19855355c2ef57b599c98a7e8fcde: Status 404 returned error can't find the container with id dc0f0709b14a6732b97ce6a0d1f953e1eca19855355c2ef57b599c98a7e8fcde
Sep 19 09:07:33.197241 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.197173    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nl6b5"]
Sep 19 09:07:33.200285 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:33.200260    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3da542d9_bdb7_460e_a8dc_da326cf2cec0.slice/crio-fa8c19dec0a64ac7822c32f7ac0a4609d9a5da15ba6d74b3c586ea1067ae8e6f WatchSource:0}: Error finding container fa8c19dec0a64ac7822c32f7ac0a4609d9a5da15ba6d74b3c586ea1067ae8e6f: Status 404 returned error can't find the container with id fa8c19dec0a64ac7822c32f7ac0a4609d9a5da15ba6d74b3c586ea1067ae8e6f
Sep 19 09:07:33.373268 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.373236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:33.521612 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.521530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:33.572983 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.572944    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:33.573170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.573003    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:33.575529 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.575506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a06e1ca-d6f0-4504-bef6-b573918caa98-metrics-tls\") pod \"dns-default-d7jp4\" (UID: \"0a06e1ca-d6f0-4504-bef6-b573918caa98\") " pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:33.575611 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.575580    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"image-registry-57fcc9f7b7-ksdtf\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") " pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:33.674041 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.673998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:33.674251 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:33.674234    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:33.674319 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:33.674301    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls podName:6e7c3e3d-26d5-4268-88fa-70e24f64d7ed nodeName:}" failed. No retries permitted until 2026-09-19 09:07:49.674280393 +0000 UTC m=+83.361085688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-98ptl" (UID: "6e7c3e3d-26d5-4268-88fa-70e24f64d7ed") : secret "cluster-monitoring-operator-tls" not found
Sep 19 09:07:33.765007 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.764958    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:33.809647 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.809566    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2nnxz\""
Sep 19 09:07:33.817380 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.817133    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:33.836013 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.835985    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xchdv\""
Sep 19 09:07:33.844011 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.843938    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:33.976526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:33.976492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:33.976702 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:33.976651    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 19 09:07:33.976754 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:07:33.976728    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert podName:a2870d9c-99a3-4533-a2d7-9fe87b230b6b nodeName:}" failed. No retries permitted until 2026-09-19 09:07:49.976708514 +0000 UTC m=+83.663513789 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-k95hw" (UID: "a2870d9c-99a3-4533-a2d7-9fe87b230b6b") : secret "networking-console-plugin-cert" not found
Sep 19 09:07:34.134604 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.134564    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kr42v" event={"ID":"60bb9f92-155a-4f69-b9d8-845edce54d11","Type":"ContainerStarted","Data":"dc0f0709b14a6732b97ce6a0d1f953e1eca19855355c2ef57b599c98a7e8fcde"}
Sep 19 09:07:34.136232 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.136181    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nl6b5" event={"ID":"3da542d9-bdb7-460e-a8dc-da326cf2cec0","Type":"ContainerStarted","Data":"fa8c19dec0a64ac7822c32f7ac0a4609d9a5da15ba6d74b3c586ea1067ae8e6f"}
Sep 19 09:07:34.142483 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.142272    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"]
Sep 19 09:07:34.144941 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:34.144907    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f45920_e15b_42e9_80bc_47f18cc1de7f.slice/crio-e1510597321c0e2fb3ae8fd4cf54d83865ffb1d275474c24cd9d8677bf84c049 WatchSource:0}: Error finding container e1510597321c0e2fb3ae8fd4cf54d83865ffb1d275474c24cd9d8677bf84c049: Status 404 returned error can't find the container with id e1510597321c0e2fb3ae8fd4cf54d83865ffb1d275474c24cd9d8677bf84c049
Sep 19 09:07:34.167797 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.167700    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-d7jp4"]
Sep 19 09:07:34.179449 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:34.179421    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a06e1ca_d6f0_4504_bef6_b573918caa98.slice/crio-bf317033ef1e605536643c808fb95122c2b4947173d9dc068f8c0cb84832ca79 WatchSource:0}: Error finding container bf317033ef1e605536643c808fb95122c2b4947173d9dc068f8c0cb84832ca79: Status 404 returned error can't find the container with id bf317033ef1e605536643c808fb95122c2b4947173d9dc068f8c0cb84832ca79
Sep 19 09:07:34.212613 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.212578    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:34.589209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.589125    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:34.780601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.780298    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:34.988749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:34.988696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:35.141753 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:35.141686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7jp4" event={"ID":"0a06e1ca-d6f0-4504-bef6-b573918caa98","Type":"ContainerStarted","Data":"bf317033ef1e605536643c808fb95122c2b4947173d9dc068f8c0cb84832ca79"}
Sep 19 09:07:35.144420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:35.143639    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" event={"ID":"c7f45920-e15b-42e9-80bc-47f18cc1de7f","Type":"ContainerStarted","Data":"913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876"}
Sep 19 09:07:35.144420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:35.143685    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" event={"ID":"c7f45920-e15b-42e9-80bc-47f18cc1de7f","Type":"ContainerStarted","Data":"e1510597321c0e2fb3ae8fd4cf54d83865ffb1d275474c24cd9d8677bf84c049"}
Sep 19 09:07:35.144420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:35.144380    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:07:35.406147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:35.406069    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:35.627532 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:35.627494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:35.798006 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:35.797925    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:36.187450 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:36.187409    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:36.613493 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:36.613418    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:36.768669 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:36.768633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:37.035275 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:37.035236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:37.228816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:37.228766    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:37.561511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:37.561467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:38.188394 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.188361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:38.590515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.590451    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:38.720383 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.720292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:38.720383 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.720340    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:38.720840 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.720823    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c83865a1-9ca7-46ec-9edf-75a9899f261a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:38.722949 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.722924    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c83865a1-9ca7-46ec-9edf-75a9899f261a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7m7zt\" (UID: \"c83865a1-9ca7-46ec-9edf-75a9899f261a\") " pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:38.870124 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.870071    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-w276g\""
Sep 19 09:07:38.876072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.876056    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7m7zt"
Sep 19 09:07:38.990072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:38.989995    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:39.160023 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.159984    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kr42v" event={"ID":"60bb9f92-155a-4f69-b9d8-845edce54d11","Type":"ContainerStarted","Data":"198eea4e858c28a17b95c521e2314e40231c82a4086be48e418ff54b1bc76ac7"}
Sep 19 09:07:39.160998 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.160970    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:39.161759 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.161742    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nl6b5" event={"ID":"3da542d9-bdb7-460e-a8dc-da326cf2cec0","Type":"ContainerStarted","Data":"488ec356b995679bfb7c81b999acfabf89a6c9368830eb2e929e1053f068d699"}
Sep 19 09:07:39.161836 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.161794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:07:39.163653 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.163629    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7jp4" event={"ID":"0a06e1ca-d6f0-4504-bef6-b573918caa98","Type":"ContainerStarted","Data":"1760d255cc745abf7fcc52dc69ef6013a1843792a4693f2cb46e1535fbd57765"}
Sep 19 09:07:39.163759 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.163673    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-d7jp4" event={"ID":"0a06e1ca-d6f0-4504-bef6-b573918caa98","Type":"ContainerStarted","Data":"12ba8f068e1a1483faffd4332c3dfd3633fda5f4e1c78306f990796959f77d0f"}
Sep 19 09:07:39.163759 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.163708    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:39.180415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.180367    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" podStartSLOduration=71.180352564 podStartE2EDuration="1m11.180352564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:35.180822527 +0000 UTC m=+68.867627824" watchObservedRunningTime="2026-09-19 09:07:39.180352564 +0000 UTC m=+72.867157860"
Sep 19 09:07:39.180741 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.180727    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7m7zt"]
Sep 19 09:07:39.183702 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:39.183678    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83865a1_9ca7_46ec_9edf_75a9899f261a.slice/crio-c77ca6ecb3aafbf83ada3bd8efc43affd45d438276089afc40760a452374ae38 WatchSource:0}: Error finding container c77ca6ecb3aafbf83ada3bd8efc43affd45d438276089afc40760a452374ae38: Status 404 returned error can't find the container with id c77ca6ecb3aafbf83ada3bd8efc43affd45d438276089afc40760a452374ae38
Sep 19 09:07:39.203168 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.203123    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kr42v" podStartSLOduration=65.977178458 podStartE2EDuration="1m11.203088118s" totalImagesPullingTime="5.22590966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:39.202769192 +0000 UTC m=+72.889574489" watchObservedRunningTime="2026-09-19 09:07:39.203088118 +0000 UTC m=+72.889893415"
Sep 19 09:07:39.297174 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.297127    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-d7jp4" podStartSLOduration=34.133398565 podStartE2EDuration="38.297094561s" totalImagesPullingTime="4.163695996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:39.271187723 +0000 UTC m=+72.957993021" watchObservedRunningTime="2026-09-19 09:07:39.297094561 +0000 UTC m=+72.983899870"
Sep 19 09:07:39.297335 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.297251    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nl6b5" podStartSLOduration=66.155560485 podStartE2EDuration="1m11.297247358s" totalImagesPullingTime="5.141686873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:39.296281636 +0000 UTC m=+72.983086942" watchObservedRunningTime="2026-09-19 09:07:39.297247358 +0000 UTC m=+72.984052646"
Sep 19 09:07:39.427293 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.427261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:39.845080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.845046    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:39.989179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:39.989145    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:40.168710 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:40.168666    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7m7zt" event={"ID":"c83865a1-9ca7-46ec-9edf-75a9899f261a","Type":"ContainerStarted","Data":"fcdad33ab1e3738cc93ae404ae7c1d4a1b796efceca1d251427102af7951248c"}
Sep 19 09:07:40.168887 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:40.168757    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7m7zt" event={"ID":"c83865a1-9ca7-46ec-9edf-75a9899f261a","Type":"ContainerStarted","Data":"c77ca6ecb3aafbf83ada3bd8efc43affd45d438276089afc40760a452374ae38"}
Sep 19 09:07:41.174746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:41.174707    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7m7zt" event={"ID":"c83865a1-9ca7-46ec-9edf-75a9899f261a","Type":"ContainerStarted","Data":"ca3077b7eef9699b3041bcb164e459b78a125fecde5a5a995aed0ccac347703c"}
Sep 19 09:07:42.180494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:42.180456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7m7zt" event={"ID":"c83865a1-9ca7-46ec-9edf-75a9899f261a","Type":"ContainerStarted","Data":"7c62a569f7e48cac68be294ea7f506663fc74b678cd7df7e86aa137958a05347"}
Sep 19 09:07:44.778425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:44.778395    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:44.900392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:44.900365    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:45.053669 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:45.053587    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:45.561782 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:45.561751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:45.745163 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:45.745131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:45.869206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:45.869142    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:46.403671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:46.403643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:46.507124 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:46.507079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:46.670478 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:46.670396    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:46.835905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:46.835874    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:47.501001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:47.500947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:47.687171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:47.687023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:49.027788 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:49.027755    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:49.170792 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:49.170766    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-d7jp4"
Sep 19 09:07:49.232156 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:49.232090    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7m7zt" podStartSLOduration=24.907058819 podStartE2EDuration="27.232074855s" totalImagesPullingTime="2.325016036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:07:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:42.210000514 +0000 UTC m=+75.896805812" watchObservedRunningTime="2026-09-19 09:07:49.232074855 +0000 UTC m=+82.918880151"
Sep 19 09:07:49.708078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:49.708032    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:49.710553 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:49.710507    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6e7c3e3d-26d5-4268-88fa-70e24f64d7ed-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-98ptl\" (UID: \"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:49.952017 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:49.951985    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-96lgx\""
Sep 19 09:07:49.959588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:49.959540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"
Sep 19 09:07:50.011276 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:50.011244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:50.014257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:50.014230    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a2870d9c-99a3-4533-a2d7-9fe87b230b6b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-k95hw\" (UID: \"a2870d9c-99a3-4533-a2d7-9fe87b230b6b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:50.179295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:50.179266    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qth4x\""
Sep 19 09:07:50.187012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:50.186990    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"
Sep 19 09:07:50.270680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:50.270650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl"]
Sep 19 09:07:50.273418 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:50.273392    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7c3e3d_26d5_4268_88fa_70e24f64d7ed.slice/crio-a93dae629c905898cf8b159894a65d94104c00ba7649106f8603d33a9c622a4f WatchSource:0}: Error finding container a93dae629c905898cf8b159894a65d94104c00ba7649106f8603d33a9c622a4f: Status 404 returned error can't find the container with id a93dae629c905898cf8b159894a65d94104c00ba7649106f8603d33a9c622a4f
Sep 19 09:07:50.485819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:50.483746    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-k95hw"]
Sep 19 09:07:50.489400 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:07:50.489372    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2870d9c_99a3_4533_a2d7_9fe87b230b6b.slice/crio-cadd8e5810216c644d90f9de6ffe56938e97cd7002fd2d82764ed8a7cb8d7dcd WatchSource:0}: Error finding container cadd8e5810216c644d90f9de6ffe56938e97cd7002fd2d82764ed8a7cb8d7dcd: Status 404 returned error can't find the container with id cadd8e5810216c644d90f9de6ffe56938e97cd7002fd2d82764ed8a7cb8d7dcd
Sep 19 09:07:51.211448 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:51.211393    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw" event={"ID":"a2870d9c-99a3-4533-a2d7-9fe87b230b6b","Type":"ContainerStarted","Data":"cadd8e5810216c644d90f9de6ffe56938e97cd7002fd2d82764ed8a7cb8d7dcd"}
Sep 19 09:07:51.212744 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:51.212716    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl" event={"ID":"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed","Type":"ContainerStarted","Data":"a93dae629c905898cf8b159894a65d94104c00ba7649106f8603d33a9c622a4f"}
Sep 19 09:07:52.661844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:52.661804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:52.950155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:52.950057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:53.174816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:53.174785    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:53.220268 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:53.220178    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl" event={"ID":"6e7c3e3d-26d5-4268-88fa-70e24f64d7ed","Type":"ContainerStarted","Data":"5edf82f2bcb968370a56a5cdca3759ca25fa60f1ca44316cb1d43b1ba653dc94"}
Sep 19 09:07:53.221564 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:53.221543    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw" event={"ID":"a2870d9c-99a3-4533-a2d7-9fe87b230b6b","Type":"ContainerStarted","Data":"5246cd719796262dd2b351af6cd6dd8e155934cecd72e44250fd9b941cbd8326"}
Sep 19 09:07:53.286779 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:53.286726    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-98ptl" podStartSLOduration=34.430107273 podStartE2EDuration="36.286708924s" totalImagesPullingTime="1.856601651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:53.258226289 +0000 UTC m=+86.945031607" watchObservedRunningTime="2026-09-19 09:07:53.286708924 +0000 UTC m=+86.973514221"
Sep 19 09:07:53.286926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:53.286899    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-k95hw" podStartSLOduration=34.650621083 podStartE2EDuration="36.286894114s" totalImagesPullingTime="1.636273031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:07:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:07:53.286063759 +0000 UTC m=+86.972869065" watchObservedRunningTime="2026-09-19 09:07:53.286894114 +0000 UTC m=+86.973699412"
Sep 19 09:07:53.822109 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:53.822057    2581 patch_prober.go:28] interesting pod/image-registry-57fcc9f7b7-ksdtf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 19 09:07:53.822463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:53.822126    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 09:07:57.155684 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:57.155645    2581 patch_prober.go:28] interesting pod/image-registry-57fcc9f7b7-ksdtf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 19 09:07:57.156051 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:57.155696    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 09:07:58.529595 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:58.529564    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:58.640522 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:58.640488    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:07:59.910975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:07:59.910941    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:00.194542 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:00.194448    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:00.311121 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:00.311065    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:01.311039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:01.311005    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:03.332785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:03.332752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:03.458536 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:03.458485    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:03.821750 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:03.821718    2581 patch_prober.go:28] interesting pod/image-registry-57fcc9f7b7-ksdtf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 19 09:08:03.821899 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:03.821767    2581 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 09:08:04.148862 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.148832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:04.519824 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.519737    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:04.955248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.955212    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"]
Sep 19 09:08:04.959895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.959879    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:04.991041 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.991001    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 19 09:08:04.997588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.997562    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 19 09:08:04.998633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.998613    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 19 09:08:04.999594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.999579    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 19 09:08:04.999651 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:04.999638    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 19 09:08:05.002211 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.002188    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 19 09:08:05.003319 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.003300    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 19 09:08:05.047264 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.047226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:05.062633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.062603    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"]
Sep 19 09:08:05.119869 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.119825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83952bc-f8b7-45dc-9f41-203fd611c3dc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.119869 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.119865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.120077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.119899    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.120077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.119944    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-hub\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.120077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.119964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt8xz\" (UniqueName: \"kubernetes.io/projected/f83952bc-f8b7-45dc-9f41-203fd611c3dc-kube-api-access-lt8xz\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.120077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.119980    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-ca\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.221317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.221229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83952bc-f8b7-45dc-9f41-203fd611c3dc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.221317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.221262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.221317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.221285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.221592 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.221379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-hub\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.221592 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.221420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt8xz\" (UniqueName: \"kubernetes.io/projected/f83952bc-f8b7-45dc-9f41-203fd611c3dc-kube-api-access-lt8xz\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.221592 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.221448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-ca\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.222374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.222344    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83952bc-f8b7-45dc-9f41-203fd611c3dc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.224154 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.224124    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.224154 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.224140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.224307 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.224181    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-hub\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.224307 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.224187    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f83952bc-f8b7-45dc-9f41-203fd611c3dc-ca\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.266638 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.266606    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt8xz\" (UniqueName: \"kubernetes.io/projected/f83952bc-f8b7-45dc-9f41-203fd611c3dc-kube-api-access-lt8xz\") pod \"cluster-proxy-proxy-agent-7b466f4968-xpzbz\" (UID: \"f83952bc-f8b7-45dc-9f41-203fd611c3dc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.278395 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.278373    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"
Sep 19 09:08:05.351832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.351802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:05.654151 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.654124    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz"]
Sep 19 09:08:05.657058 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:08:05.657030    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf83952bc_f8b7_45dc_9f41_203fd611c3dc.slice/crio-fee2601a4957828a7c72060b444438c766150afb65abbc8f2cd6ca0245bcf922 WatchSource:0}: Error finding container fee2601a4957828a7c72060b444438c766150afb65abbc8f2cd6ca0245bcf922: Status 404 returned error can't find the container with id fee2601a4957828a7c72060b444438c766150afb65abbc8f2cd6ca0245bcf922
Sep 19 09:08:05.829152 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:05.829122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:06.267868 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:06.267831    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz" event={"ID":"f83952bc-f8b7-45dc-9f41-203fd611c3dc","Type":"ContainerStarted","Data":"fee2601a4957828a7c72060b444438c766150afb65abbc8f2cd6ca0245bcf922"}
Sep 19 09:08:06.348824 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:06.348793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:06.615561 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:06.615452    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:07.157469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:07.157425    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:08:08.153300 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:08.153260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:08.313117 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:08.313066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:10.171841 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:10.171816    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nl6b5"
Sep 19 09:08:10.284485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:10.284446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz" event={"ID":"f83952bc-f8b7-45dc-9f41-203fd611c3dc","Type":"ContainerStarted","Data":"a28e6e08627f3e44a5e9064472d3075cc470f498508b8a37eda78b343620553d"}
Sep 19 09:08:10.284647 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:10.284492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz" event={"ID":"f83952bc-f8b7-45dc-9f41-203fd611c3dc","Type":"ContainerStarted","Data":"e97d5722947c78e6d004a8d7dba6e983893d6c8901b58fe29c640799826496af"}
Sep 19 09:08:10.284647 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:10.284508    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz" event={"ID":"f83952bc-f8b7-45dc-9f41-203fd611c3dc","Type":"ContainerStarted","Data":"348a3d5bc96a66b58511c26cb94b706e8580d5f0dbe4bb1210830959587c106e"}
Sep 19 09:08:10.387202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:10.387153    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b466f4968-xpzbz" podStartSLOduration=2.76809279 podStartE2EDuration="6.387135468s" totalImagesPullingTime="3.619042678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:10.386713207 +0000 UTC m=+104.073518508" watchObservedRunningTime="2026-09-19 09:08:10.387135468 +0000 UTC m=+104.073940766"
Sep 19 09:08:12.992015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:12.991984    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:13.117064 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:13.117030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:16.031546 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:16.031518    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:16.189645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:16.189613    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:16.441069 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:16.441030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:16.561200 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:16.561168    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:17.235143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:17.235093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:17.361344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:17.361313    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:18.985540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:18.985506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:19.230322 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:19.230286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:19.426240 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:19.423179    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:21.313264 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:21.313225    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:21.656213 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:21.656178    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:21.992376 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:21.992284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:22.010681 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.010648    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9xtr6"]
Sep 19 09:08:22.013976 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.013958    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.024175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.024153    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 09:08:22.024811 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.024795    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vz84s\""
Sep 19 09:08:22.024881 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.024798    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 09:08:22.024881 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.024831    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 09:08:22.025798 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.025782    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 09:08:22.159409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-sys\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh6mw\" (UniqueName: \"kubernetes.io/projected/2325259e-d06d-473c-a97f-d647ece2128d-kube-api-access-xh6mw\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159471    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-tls\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159498    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-wtmp\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-root\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159563    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2325259e-d06d-473c-a97f-d647ece2128d-metrics-client-ca\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159663    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159702    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-accelerators-collector-config\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.159839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.159762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-textfile\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-sys\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh6mw\" (UniqueName: \"kubernetes.io/projected/2325259e-d06d-473c-a97f-d647ece2128d-kube-api-access-xh6mw\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260238    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-tls\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260253    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-wtmp\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-root\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260292    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2325259e-d06d-473c-a97f-d647ece2128d-metrics-client-ca\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260345    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-root\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260287    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-sys\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:08:22.260389    2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:08:22.260463    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-tls podName:2325259e-d06d-473c-a97f-d647ece2128d nodeName:}" failed. No retries permitted until 2026-09-19 09:08:22.760443258 +0000 UTC m=+116.447248551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-tls") pod "node-exporter-9xtr6" (UID: "2325259e-d06d-473c-a97f-d647ece2128d") : secret "node-exporter-tls" not found
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-wtmp\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-accelerators-collector-config\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.260590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-textfile\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.261044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260862    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-textfile\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.261044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260972    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2325259e-d06d-473c-a97f-d647ece2128d-metrics-client-ca\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.261044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.260995    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-accelerators-collector-config\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.262784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.262767    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.298783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.298751    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh6mw\" (UniqueName: \"kubernetes.io/projected/2325259e-d06d-473c-a97f-d647ece2128d-kube-api-access-xh6mw\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.372511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.372474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:22.664707 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.664676    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:22.765153 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.765120    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-tls\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.767578 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.767562    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2325259e-d06d-473c-a97f-d647ece2128d-node-exporter-tls\") pod \"node-exporter-9xtr6\" (UID: \"2325259e-d06d-473c-a97f-d647ece2128d\") " pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.894089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.894052    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:22.922547 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:22.922469    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9xtr6"
Sep 19 09:08:22.933492 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:08:22.933456    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2325259e_d06d_473c_a97f_d647ece2128d.slice/crio-55f2692a207dcdde9b45a8808f16b8e6345cbf7329cfb0ac8f7237fa89d09c3b WatchSource:0}: Error finding container 55f2692a207dcdde9b45a8808f16b8e6345cbf7329cfb0ac8f7237fa89d09c3b: Status 404 returned error can't find the container with id 55f2692a207dcdde9b45a8808f16b8e6345cbf7329cfb0ac8f7237fa89d09c3b
Sep 19 09:08:23.094422 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:23.094392    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:23.330535 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:23.330453    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xtr6" event={"ID":"2325259e-d06d-473c-a97f-d647ece2128d","Type":"ContainerStarted","Data":"55f2692a207dcdde9b45a8808f16b8e6345cbf7329cfb0ac8f7237fa89d09c3b"}
Sep 19 09:08:23.526229 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:23.526197    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:23.923502 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:23.923460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:24.292372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:24.292337    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:24.336252 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:24.336212    2581 generic.go:356] "Generic (PLEG): container finished" podID="2325259e-d06d-473c-a97f-d647ece2128d" containerID="071f5c02330f6a27b1f701e1aa073e7a80b4d68126c564b6b7dcfe5e598f33ff" exitCode=0
Sep 19 09:08:24.336418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:24.336275    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xtr6" event={"ID":"2325259e-d06d-473c-a97f-d647ece2128d","Type":"ContainerDied","Data":"071f5c02330f6a27b1f701e1aa073e7a80b4d68126c564b6b7dcfe5e598f33ff"}
Sep 19 09:08:25.342373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:25.342331    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xtr6" event={"ID":"2325259e-d06d-473c-a97f-d647ece2128d","Type":"ContainerStarted","Data":"df72c1946205d4cf62588e243c1f08e8d8bce74ca7106784ff9878b0d268d0ed"}
Sep 19 09:08:25.342803 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:25.342398    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9xtr6" event={"ID":"2325259e-d06d-473c-a97f-d647ece2128d","Type":"ContainerStarted","Data":"d56dbf72dce5f574921cdcdd48566d9b47db179e2b9ea7a7f26718dd6c55769d"}
Sep 19 09:08:26.486534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:26.486502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:26.690459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:26.690422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:26.948837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:26.948802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:27.463520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:27.463490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:27.777449 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:27.777367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:27.979553 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:27.979521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:28.154409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:28.154358    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9xtr6" podStartSLOduration=5.985384359 podStartE2EDuration="7.154339792s" totalImagesPullingTime="1.097347628s" totalInitContainerRuntime="71.607805ms" isStatefulPod=true podCreationTimestamp="2026-09-19 09:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:25.442443765 +0000 UTC m=+119.129249041" watchObservedRunningTime="2026-09-19 09:08:28.154339792 +0000 UTC m=+121.841145092"
Sep 19 09:08:28.154596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:28.154521    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"]
Sep 19 09:08:28.425832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:28.425751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:28.647402 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:28.647372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:29.458972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:29.458941    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:29.693771 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:29.693739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:31.389560 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:31.389518    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:31.562562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:31.562530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:34.178219 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:34.178184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:34.361331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:34.361299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:36.767288 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:36.767250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:08:36.769707 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:36.769688    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e0c101dc-58b0-481d-9859-234a79696788-metrics-certs\") pod \"network-metrics-daemon-485pv\" (UID: \"e0c101dc-58b0-481d-9859-234a79696788\") " pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:08:37.054258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:37.054231    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fszt9\""
Sep 19 09:08:37.059005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:37.058988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-485pv"
Sep 19 09:08:37.386875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:37.386843    2581 generic.go:356] "Generic (PLEG): container finished" podID="4962a821-49df-4abb-ae80-375b5d2724fd" containerID="2e7a892ec214000d8ad5cd5a495d5b75fff2dade88446fb10c9887bad2bf098d" exitCode=0
Sep 19 09:08:37.387039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:37.386897    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-v684g" event={"ID":"4962a821-49df-4abb-ae80-375b5d2724fd","Type":"ContainerDied","Data":"2e7a892ec214000d8ad5cd5a495d5b75fff2dade88446fb10c9887bad2bf098d"}
Sep 19 09:08:37.387244 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:37.387226    2581 scope.go:117] "RemoveContainer" containerID="2e7a892ec214000d8ad5cd5a495d5b75fff2dade88446fb10c9887bad2bf098d"
Sep 19 09:08:37.407684 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:37.407661    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-485pv"]
Sep 19 09:08:37.414252 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:08:37.414229    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c101dc_58b0_481d_9859_234a79696788.slice/crio-633fc1a1636c9b66d9fa6ac03848f750959e02a68302f8331fc74ac2c94072ed WatchSource:0}: Error finding container 633fc1a1636c9b66d9fa6ac03848f750959e02a68302f8331fc74ac2c94072ed: Status 404 returned error can't find the container with id 633fc1a1636c9b66d9fa6ac03848f750959e02a68302f8331fc74ac2c94072ed
Sep 19 09:08:38.000297 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:38.000254    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:38.261968 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:38.261876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:38.392564 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:38.392519    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-485pv" event={"ID":"e0c101dc-58b0-481d-9859-234a79696788","Type":"ContainerStarted","Data":"633fc1a1636c9b66d9fa6ac03848f750959e02a68302f8331fc74ac2c94072ed"}
Sep 19 09:08:38.394917 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:38.394883    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-v684g" event={"ID":"4962a821-49df-4abb-ae80-375b5d2724fd","Type":"ContainerStarted","Data":"5ed67e6c61a98eb6d8db9c65d2dcaa66a659e1ba5424e28634861b720ba055df"}
Sep 19 09:08:39.053475 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:39.053444    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5thm_c0ad2ff9-8673-44cd-989f-893a35ed9c59/serve-healthcheck-canary/0.log"
Sep 19 09:08:39.400082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:39.400042    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-485pv" event={"ID":"e0c101dc-58b0-481d-9859-234a79696788","Type":"ContainerStarted","Data":"8a1ca2325610ac03a09dd490d4c22415f3020a18a03dd4c57ca5efa8ad99da9a"}
Sep 19 09:08:39.400082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:39.400092    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-485pv" event={"ID":"e0c101dc-58b0-481d-9859-234a79696788","Type":"ContainerStarted","Data":"fc645fb05095b3d175d2b09365d025fb4834675fb264c907031695a98725b1d6"}
Sep 19 09:08:39.440056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:39.439977    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-485pv" podStartSLOduration=130.505791362 podStartE2EDuration="2m11.43995946s" totalImagesPullingTime="934.168098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:08:39.439205911 +0000 UTC m=+133.126011211" watchObservedRunningTime="2026-09-19 09:08:39.43995946 +0000 UTC m=+133.126764757"
Sep 19 09:08:39.750561 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:39.750475    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:40.067932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:40.067850    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:46.396534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:46.396499    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:46.522591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:46.522554    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:52.453815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:52.453777    2581 generic.go:356] "Generic (PLEG): container finished" podID="3bc99bcb-8487-4ae8-89e6-de3272b082f1" containerID="9a9d36df1f721d78c84a9a0995e40deb9429162349bf982ca6b637c7c9d3474a" exitCode=0
Sep 19 09:08:52.454218 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:52.453849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b" event={"ID":"3bc99bcb-8487-4ae8-89e6-de3272b082f1","Type":"ContainerDied","Data":"9a9d36df1f721d78c84a9a0995e40deb9429162349bf982ca6b637c7c9d3474a"}
Sep 19 09:08:52.454218 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:52.454189    2581 scope.go:117] "RemoveContainer" containerID="9a9d36df1f721d78c84a9a0995e40deb9429162349bf982ca6b637c7c9d3474a"
Sep 19 09:08:53.173788 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.173734    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerName="registry" containerID="cri-o://913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876" gracePeriod=30
Sep 19 09:08:53.441001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.440974    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:08:53.462418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.462383    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-p2m8b" event={"ID":"3bc99bcb-8487-4ae8-89e6-de3272b082f1","Type":"ContainerStarted","Data":"d98a20ffdb2f0a8328849e2f5c0817b1a191ffac158d5d42504027319c00e301"}
Sep 19 09:08:53.463745 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.463721    2581 generic.go:356] "Generic (PLEG): container finished" podID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerID="913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876" exitCode=0
Sep 19 09:08:53.463867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.463749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" event={"ID":"c7f45920-e15b-42e9-80bc-47f18cc1de7f","Type":"ContainerDied","Data":"913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876"}
Sep 19 09:08:53.463867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.463793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf" event={"ID":"c7f45920-e15b-42e9-80bc-47f18cc1de7f","Type":"ContainerDied","Data":"e1510597321c0e2fb3ae8fd4cf54d83865ffb1d275474c24cd9d8677bf84c049"}
Sep 19 09:08:53.463867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.463762    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"
Sep 19 09:08:53.463867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.463812    2581 scope.go:117] "RemoveContainer" containerID="913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876"
Sep 19 09:08:53.472611 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.472589    2581 scope.go:117] "RemoveContainer" containerID="913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876"
Sep 19 09:08:53.472945 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:08:53.472920    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876\": container with ID starting with 913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876 not found: ID does not exist" containerID="913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876"
Sep 19 09:08:53.473015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.472959    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876"} err="failed to get container status \"913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876\": rpc error: code = NotFound desc = could not find container \"913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876\": container with ID starting with 913edd18a530b603f6a5e28fc7033c500ec64ea503c5f32f5cc81080dc957876 not found: ID does not exist"
Sep 19 09:08:53.495966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.495935    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-installation-pull-secrets\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.495975    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-certificates\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496006    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7f45920-e15b-42e9-80bc-47f18cc1de7f-ca-trust-extracted\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496040    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496062    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf8nc\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-kube-api-access-pf8nc\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496288 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496084    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-trusted-ca\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496288 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496122    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-image-registry-private-configuration\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496288 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496146    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-bound-sa-token\") pod \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\" (UID: \"c7f45920-e15b-42e9-80bc-47f18cc1de7f\") "
Sep 19 09:08:53.496613 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496498    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:08:53.496985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.496719    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:08:53.498952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.498922    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:08:53.499068 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.499007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-kube-api-access-pf8nc" (OuterVolumeSpecName: "kube-api-access-pf8nc") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "kube-api-access-pf8nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:08:53.499152 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.499132    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:08:53.499205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.499150    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:08:53.499418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.499397    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:08:53.505615 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.505559    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7f45920-e15b-42e9-80bc-47f18cc1de7f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c7f45920-e15b-42e9-80bc-47f18cc1de7f" (UID: "c7f45920-e15b-42e9-80bc-47f18cc1de7f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:08:53.597566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597530    2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-installation-pull-secrets\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.597566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597562    2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-certificates\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.597566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597573    2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c7f45920-e15b-42e9-80bc-47f18cc1de7f-ca-trust-extracted\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.597809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597582    2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-registry-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.597809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597590    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf8nc\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-kube-api-access-pf8nc\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.597809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597598    2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c7f45920-e15b-42e9-80bc-47f18cc1de7f-trusted-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.597809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597607    2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c7f45920-e15b-42e9-80bc-47f18cc1de7f-image-registry-private-configuration\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.597809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.597615    2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c7f45920-e15b-42e9-80bc-47f18cc1de7f-bound-sa-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:08:53.797961 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.797923    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"]
Sep 19 09:08:53.808762 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:53.808735    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57fcc9f7b7-ksdtf"]
Sep 19 09:08:54.124705 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:54.124589    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:54.336774 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:54.336735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:54.498411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:54.498362    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:08:54.829815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:08:54.829734    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" path="/var/lib/kubelet/pods/c7f45920-e15b-42e9-80bc-47f18cc1de7f/volumes"
Sep 19 09:09:11.767633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:09:11.767592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:09:12.003049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:09:12.003018    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:08.674813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:10:08.674777    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:10:08.792610 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:10:08.792576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:11:19.816289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:11:19.816208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:11:26.744533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:11:26.744501    2581 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 09:13:27.266426 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:27.266377    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:28.755014 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:28.754968    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:30.518045 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:30.518012    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:30.710598 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:30.710556    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:30.913951 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:30.913908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:33.951735 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:33.951693    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:35.738886 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:35.738853    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:35.820373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:35.820336    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:46.613721 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:46.613681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:13:46.794425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:13:46.794397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:43.778514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:43.778469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:43.902670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:43.902622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:44.834129 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:44.834060    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:44.902240 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:44.902207    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:51.154629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:51.154579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:55.708727 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:55.708688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:55.814475 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:55.814435    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:14:55.914969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:14:55.914933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:15:01.355798 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:15:01.355761    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:15:01.456535 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:15:01.456502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:15:40.689620 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:15:40.689585    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:15:40.800912 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:15:40.800869    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:16:31.213362 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:16:31.213323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:16:31.268525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:16:31.268487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:30.367238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:30.367199    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:31.525003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:31.524966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:32.906874 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:32.906832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:35.138680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:35.138637    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:35.238564 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:35.238529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:36.660871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:36.660838    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:36.761132 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:36.761078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:36.886150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:36.886092    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:36.987269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:36.987176    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:37.547364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:37.547331    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:37.652186 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:37.652156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:38.162221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.162188    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"]
Sep 19 09:17:38.162593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.162485    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerName="registry"
Sep 19 09:17:38.162593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.162497    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerName="registry"
Sep 19 09:17:38.162593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.162548    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7f45920-e15b-42e9-80bc-47f18cc1de7f" containerName="registry"
Sep 19 09:17:38.165362 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.165347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.167982 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.167959    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 09:17:38.168163 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.167985    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kklnq\""
Sep 19 09:17:38.168163 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.168016    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 09:17:38.174596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.174571    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"]
Sep 19 09:17:38.302286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.302252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.302463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.302299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trx4n\" (UniqueName: \"kubernetes.io/projected/0819823e-b364-4f27-add4-9916cdb49644-kube-api-access-trx4n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.302463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.302325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.403320 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.403283    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.403512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.403334    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trx4n\" (UniqueName: \"kubernetes.io/projected/0819823e-b364-4f27-add4-9916cdb49644-kube-api-access-trx4n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.403512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.403369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.403667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.403646    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.403743 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.403723    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.412001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.411973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trx4n\" (UniqueName: \"kubernetes.io/projected/0819823e-b364-4f27-add4-9916cdb49644-kube-api-access-trx4n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.474767 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.474689    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:38.748596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.748515    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"]
Sep 19 09:17:38.751735 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:17:38.751697    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0819823e_b364_4f27_add4_9916cdb49644.slice/crio-befcbc4d5d99819b80948ac823717cbb8149af54985f66769c1a6f7811c21ad4 WatchSource:0}: Error finding container befcbc4d5d99819b80948ac823717cbb8149af54985f66769c1a6f7811c21ad4: Status 404 returned error can't find the container with id befcbc4d5d99819b80948ac823717cbb8149af54985f66769c1a6f7811c21ad4
Sep 19 09:17:38.753658 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:38.753638    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 09:17:39.109632 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:39.109542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:39.306789 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:39.306750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:39.317321 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:39.317290    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd" event={"ID":"0819823e-b364-4f27-add4-9916cdb49644","Type":"ContainerStarted","Data":"befcbc4d5d99819b80948ac823717cbb8149af54985f66769c1a6f7811c21ad4"}
Sep 19 09:17:41.923442 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:41.923373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:42.026384 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:42.026327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:45.341177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:45.341143    2581 generic.go:356] "Generic (PLEG): container finished" podID="0819823e-b364-4f27-add4-9916cdb49644" containerID="2e23757182b271e7a79ad22330b61e32fa8a088f19986ca8db572cca76c17472" exitCode=0
Sep 19 09:17:45.341564 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:45.341202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd" event={"ID":"0819823e-b364-4f27-add4-9916cdb49644","Type":"ContainerDied","Data":"2e23757182b271e7a79ad22330b61e32fa8a088f19986ca8db572cca76c17472"}
Sep 19 09:17:47.119854 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.119820    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:47.371992 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.371924    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-525qg"]
Sep 19 09:17:47.376209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.376188    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.378485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.378465    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-w2v9k\""
Sep 19 09:17:47.378597 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.378571    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 19 09:17:47.379337 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.379322    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 19 09:17:47.385495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.385472    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-525qg"]
Sep 19 09:17:47.474589 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.474560    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5k85\" (UniqueName: \"kubernetes.io/projected/856f17c9-8041-4027-865d-8d3f63c101eb-kube-api-access-x5k85\") pod \"cert-manager-55b4d954d5-525qg\" (UID: \"856f17c9-8041-4027-865d-8d3f63c101eb\") " pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.474733 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.474597    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/856f17c9-8041-4027-865d-8d3f63c101eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-525qg\" (UID: \"856f17c9-8041-4027-865d-8d3f63c101eb\") " pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.575423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.575375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5k85\" (UniqueName: \"kubernetes.io/projected/856f17c9-8041-4027-865d-8d3f63c101eb-kube-api-access-x5k85\") pod \"cert-manager-55b4d954d5-525qg\" (UID: \"856f17c9-8041-4027-865d-8d3f63c101eb\") " pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.575423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.575421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/856f17c9-8041-4027-865d-8d3f63c101eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-525qg\" (UID: \"856f17c9-8041-4027-865d-8d3f63c101eb\") " pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.587899 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.587863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5k85\" (UniqueName: \"kubernetes.io/projected/856f17c9-8041-4027-865d-8d3f63c101eb-kube-api-access-x5k85\") pod \"cert-manager-55b4d954d5-525qg\" (UID: \"856f17c9-8041-4027-865d-8d3f63c101eb\") " pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.589036 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.589003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/856f17c9-8041-4027-865d-8d3f63c101eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-525qg\" (UID: \"856f17c9-8041-4027-865d-8d3f63c101eb\") " pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.685936 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.685888    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-525qg"
Sep 19 09:17:47.956271 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:47.956243    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-525qg"]
Sep 19 09:17:47.958581 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:17:47.958560    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod856f17c9_8041_4027_865d_8d3f63c101eb.slice/crio-7eef79cee2e5a45bc96de83c52ed8ef92b65f5a59c43f539b0b2f8e375803ca2 WatchSource:0}: Error finding container 7eef79cee2e5a45bc96de83c52ed8ef92b65f5a59c43f539b0b2f8e375803ca2: Status 404 returned error can't find the container with id 7eef79cee2e5a45bc96de83c52ed8ef92b65f5a59c43f539b0b2f8e375803ca2
Sep 19 09:17:48.354919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:48.354828    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-525qg" event={"ID":"856f17c9-8041-4027-865d-8d3f63c101eb","Type":"ContainerStarted","Data":"7eef79cee2e5a45bc96de83c52ed8ef92b65f5a59c43f539b0b2f8e375803ca2"}
Sep 19 09:17:48.356671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:48.356645    2581 generic.go:356] "Generic (PLEG): container finished" podID="0819823e-b364-4f27-add4-9916cdb49644" containerID="82f04f1e10c3a399af4062cf0f73053bae5253acb0303175f78b7166632a661e" exitCode=0
Sep 19 09:17:48.356769 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:48.356722    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd" event={"ID":"0819823e-b364-4f27-add4-9916cdb49644","Type":"ContainerDied","Data":"82f04f1e10c3a399af4062cf0f73053bae5253acb0303175f78b7166632a661e"}
Sep 19 09:17:52.378746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:52.378707    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-525qg" event={"ID":"856f17c9-8041-4027-865d-8d3f63c101eb","Type":"ContainerStarted","Data":"ca10f2cb7a365dc26ac1d5ad03366f9222627196e19dfd41718ee676aa34d04f"}
Sep 19 09:17:52.403196 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:52.403135    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-525qg" podStartSLOduration=1.3503117310000001 podStartE2EDuration="5.403115789s" totalImagesPullingTime="4.052804058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:17:52.401279362 +0000 UTC m=+686.088084675" watchObservedRunningTime="2026-09-19 09:17:52.403115789 +0000 UTC m=+686.089921082"
Sep 19 09:17:56.397151 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:56.397091    2581 generic.go:356] "Generic (PLEG): container finished" podID="0819823e-b364-4f27-add4-9916cdb49644" containerID="0e669b276e68b8dcfcb5abf3844dc36afaf4ccc1c7eede373ed019c501078362" exitCode=0
Sep 19 09:17:56.397151 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:56.397131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd" event={"ID":"0819823e-b364-4f27-add4-9916cdb49644","Type":"ContainerDied","Data":"0e669b276e68b8dcfcb5abf3844dc36afaf4ccc1c7eede373ed019c501078362"}
Sep 19 09:17:57.529065 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.529035    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:57.554234 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.554203    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trx4n\" (UniqueName: \"kubernetes.io/projected/0819823e-b364-4f27-add4-9916cdb49644-kube-api-access-trx4n\") pod \"0819823e-b364-4f27-add4-9916cdb49644\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") "
Sep 19 09:17:57.554418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.554249    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-bundle\") pod \"0819823e-b364-4f27-add4-9916cdb49644\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") "
Sep 19 09:17:57.554418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.554320    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-util\") pod \"0819823e-b364-4f27-add4-9916cdb49644\" (UID: \"0819823e-b364-4f27-add4-9916cdb49644\") "
Sep 19 09:17:57.554664 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.554639    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-bundle" (OuterVolumeSpecName: "bundle") pod "0819823e-b364-4f27-add4-9916cdb49644" (UID: "0819823e-b364-4f27-add4-9916cdb49644"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:17:57.556718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.556691    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0819823e-b364-4f27-add4-9916cdb49644-kube-api-access-trx4n" (OuterVolumeSpecName: "kube-api-access-trx4n") pod "0819823e-b364-4f27-add4-9916cdb49644" (UID: "0819823e-b364-4f27-add4-9916cdb49644"). InnerVolumeSpecName "kube-api-access-trx4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:17:57.562943 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.562911    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-util" (OuterVolumeSpecName: "util") pod "0819823e-b364-4f27-add4-9916cdb49644" (UID: "0819823e-b364-4f27-add4-9916cdb49644"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:17:57.655316 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.655229    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trx4n\" (UniqueName: \"kubernetes.io/projected/0819823e-b364-4f27-add4-9916cdb49644-kube-api-access-trx4n\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:17:57.655316 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.655259    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-bundle\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:17:57.655316 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:57.655268    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0819823e-b364-4f27-add4-9916cdb49644-util\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:17:58.406354 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:58.406318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd" event={"ID":"0819823e-b364-4f27-add4-9916cdb49644","Type":"ContainerDied","Data":"befcbc4d5d99819b80948ac823717cbb8149af54985f66769c1a6f7811c21ad4"}
Sep 19 09:17:58.406533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:58.406374    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="befcbc4d5d99819b80948ac823717cbb8149af54985f66769c1a6f7811c21ad4"
Sep 19 09:17:58.406533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:58.406326    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqhgxd"
Sep 19 09:17:58.981521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:58.981481    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:17:59.048700 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:17:59.048664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:00.688352 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:00.688320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:00.788627 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:00.788592    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:02.602555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:02.602487    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:02.702578 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:02.702536    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:04.547054 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:04.547011    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:04.652232 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:04.652196    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:06.934666 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:06.934629    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:07.035685 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:07.035640    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:08.443604 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443576    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"]
Sep 19 09:18:08.443981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443850    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0819823e-b364-4f27-add4-9916cdb49644" containerName="pull"
Sep 19 09:18:08.443981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443862    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0819823e-b364-4f27-add4-9916cdb49644" containerName="pull"
Sep 19 09:18:08.443981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443874    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0819823e-b364-4f27-add4-9916cdb49644" containerName="util"
Sep 19 09:18:08.443981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443879    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0819823e-b364-4f27-add4-9916cdb49644" containerName="util"
Sep 19 09:18:08.443981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443888    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0819823e-b364-4f27-add4-9916cdb49644" containerName="extract"
Sep 19 09:18:08.443981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443893    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0819823e-b364-4f27-add4-9916cdb49644" containerName="extract"
Sep 19 09:18:08.443981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.443934    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0819823e-b364-4f27-add4-9916cdb49644" containerName="extract"
Sep 19 09:18:08.449465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.449436    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.452047 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.452026    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kklnq\""
Sep 19 09:18:08.452185 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.452088    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 09:18:08.452742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.452729    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 09:18:08.455238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.455214    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"]
Sep 19 09:18:08.532872 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.532840    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.533015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.532885    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c7mp\" (UniqueName: \"kubernetes.io/projected/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-kube-api-access-9c7mp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.533015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.532976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.610927 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.610875    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:08.634280 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.634247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.634447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.634315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.634447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.634355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c7mp\" (UniqueName: \"kubernetes.io/projected/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-kube-api-access-9c7mp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.634675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.634652    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.634675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.634667    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.643064 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.643038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c7mp\" (UniqueName: \"kubernetes.io/projected/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-kube-api-access-9c7mp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:08.759659 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:08.759568    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:09.032119 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:09.032010    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"]
Sep 19 09:18:09.034797 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:18:09.034768    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf54c6be6_e8f7_4796_8fd7_dce9b2b3c6e2.slice/crio-356313de4d62a62afe407887031a69853b8bd5b5357f4575f52078046221c14f WatchSource:0}: Error finding container 356313de4d62a62afe407887031a69853b8bd5b5357f4575f52078046221c14f: Status 404 returned error can't find the container with id 356313de4d62a62afe407887031a69853b8bd5b5357f4575f52078046221c14f
Sep 19 09:18:09.448419 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:09.448386    2581 generic.go:356] "Generic (PLEG): container finished" podID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerID="6c21ac365ef23c6dd8d6c7e25a63543a042801885cd62f67eaff98ced97c7a76" exitCode=0
Sep 19 09:18:09.448808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:09.448427    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d" event={"ID":"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2","Type":"ContainerDied","Data":"6c21ac365ef23c6dd8d6c7e25a63543a042801885cd62f67eaff98ced97c7a76"}
Sep 19 09:18:09.448808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:09.448460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d" event={"ID":"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2","Type":"ContainerStarted","Data":"356313de4d62a62afe407887031a69853b8bd5b5357f4575f52078046221c14f"}
Sep 19 09:18:10.454176 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:10.454143    2581 generic.go:356] "Generic (PLEG): container finished" podID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerID="1af8ee49d7c4de515abb6b6fc02fd9e2ce7f84d8a8c1a49e8a7757048257c8ba" exitCode=0
Sep 19 09:18:10.454564 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:10.454208    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d" event={"ID":"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2","Type":"ContainerDied","Data":"1af8ee49d7c4de515abb6b6fc02fd9e2ce7f84d8a8c1a49e8a7757048257c8ba"}
Sep 19 09:18:11.229991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:11.229957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:11.330697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:11.330659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:11.460275 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:11.460243    2581 generic.go:356] "Generic (PLEG): container finished" podID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerID="e056b0534debd101b45c3ebf013737eed987fd1c79fa02bb56821cb08c46600e" exitCode=0
Sep 19 09:18:11.460275 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:11.460281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d" event={"ID":"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2","Type":"ContainerDied","Data":"e056b0534debd101b45c3ebf013737eed987fd1c79fa02bb56821cb08c46600e"}
Sep 19 09:18:12.587418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.587389    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:12.663787 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.663749    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-util\") pod \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") "
Sep 19 09:18:12.663964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.663836    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-bundle\") pod \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") "
Sep 19 09:18:12.663964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.663869    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c7mp\" (UniqueName: \"kubernetes.io/projected/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-kube-api-access-9c7mp\") pod \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\" (UID: \"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2\") "
Sep 19 09:18:12.664720 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.664635    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-bundle" (OuterVolumeSpecName: "bundle") pod "f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" (UID: "f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:18:12.666236 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.666210    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-kube-api-access-9c7mp" (OuterVolumeSpecName: "kube-api-access-9c7mp") pod "f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" (UID: "f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2"). InnerVolumeSpecName "kube-api-access-9c7mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:18:12.671875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.671845    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-util" (OuterVolumeSpecName: "util") pod "f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" (UID: "f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:18:12.764954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.764859    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-util\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:18:12.764954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.764901    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-bundle\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:18:12.764954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:12.764911    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c7mp\" (UniqueName: \"kubernetes.io/projected/f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2-kube-api-access-9c7mp\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:18:13.469968 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:13.469937    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d"
Sep 19 09:18:13.469968 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:13.469955    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxlb7d" event={"ID":"f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2","Type":"ContainerDied","Data":"356313de4d62a62afe407887031a69853b8bd5b5357f4575f52078046221c14f"}
Sep 19 09:18:13.470231 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:13.469994    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="356313de4d62a62afe407887031a69853b8bd5b5357f4575f52078046221c14f"
Sep 19 09:18:19.077894 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.077861    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"]
Sep 19 09:18:19.078410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.078249    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerName="util"
Sep 19 09:18:19.078410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.078268    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerName="util"
Sep 19 09:18:19.078410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.078286    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerName="pull"
Sep 19 09:18:19.078410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.078293    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerName="pull"
Sep 19 09:18:19.078410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.078301    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerName="extract"
Sep 19 09:18:19.078410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.078309    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerName="extract"
Sep 19 09:18:19.078410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.078381    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f54c6be6-e8f7-4796-8fd7-dce9b2b3c6e2" containerName="extract"
Sep 19 09:18:19.084135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.084092    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.086627 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.086600    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 09:18:19.086814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.086801    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 09:18:19.087466 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.087450    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-kklnq\""
Sep 19 09:18:19.092775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.092749    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"]
Sep 19 09:18:19.212464 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.212422    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ww2g\" (UniqueName: \"kubernetes.io/projected/d7af1339-caf4-4bbf-acf6-84b617dd3416-kube-api-access-8ww2g\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.212647 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.212485    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.212647 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.212539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.313450 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.313398    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ww2g\" (UniqueName: \"kubernetes.io/projected/d7af1339-caf4-4bbf-acf6-84b617dd3416-kube-api-access-8ww2g\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.313635 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.313478    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.313635 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.313522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.313913 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.313886    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.313991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.313894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.323724 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.323704    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ww2g\" (UniqueName: \"kubernetes.io/projected/d7af1339-caf4-4bbf-acf6-84b617dd3416-kube-api-access-8ww2g\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.393704 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.393670    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:19.667250 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:19.667170    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"]
Sep 19 09:18:19.670359 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:18:19.670330    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7af1339_caf4_4bbf_acf6_84b617dd3416.slice/crio-ddc2b763b6f792c857da21650fc6aa8784c043b55b50835b5d8e17200de5dd89 WatchSource:0}: Error finding container ddc2b763b6f792c857da21650fc6aa8784c043b55b50835b5d8e17200de5dd89: Status 404 returned error can't find the container with id ddc2b763b6f792c857da21650fc6aa8784c043b55b50835b5d8e17200de5dd89
Sep 19 09:18:20.348985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:20.348952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:20.454608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:20.454570    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:20.503035 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:20.502995    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerID="9f13e02569a07f7907d4e1af3afae11121ba21232b32d2fc418491cca0f4947a" exitCode=0
Sep 19 09:18:20.503214 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:20.503039    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t" event={"ID":"d7af1339-caf4-4bbf-acf6-84b617dd3416","Type":"ContainerDied","Data":"9f13e02569a07f7907d4e1af3afae11121ba21232b32d2fc418491cca0f4947a"}
Sep 19 09:18:20.503214 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:20.503134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t" event={"ID":"d7af1339-caf4-4bbf-acf6-84b617dd3416","Type":"ContainerStarted","Data":"ddc2b763b6f792c857da21650fc6aa8784c043b55b50835b5d8e17200de5dd89"}
Sep 19 09:18:20.979516 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:20.979477    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:21.235729 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.235694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:21.334619 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.334584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:21.388067 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.388035    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"]
Sep 19 09:18:21.391272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.391256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.393970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.393949    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 09:18:21.393970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.393963    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 09:18:21.394743 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.394727    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 09:18:21.394901 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.394884    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7pvgm\""
Sep 19 09:18:21.395052 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.395036    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 09:18:21.416059 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.416031    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"]
Sep 19 09:18:21.431610 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.431583    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8b98810-667c-4ad9-8724-7c64dd6be296-tmp\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.431746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.431618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8b98810-667c-4ad9-8724-7c64dd6be296-manifests\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.431746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.431646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j7g8\" (UniqueName: \"kubernetes.io/projected/c8b98810-667c-4ad9-8724-7c64dd6be296-kube-api-access-2j7g8\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.431746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.431685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b98810-667c-4ad9-8724-7c64dd6be296-webhook-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.431746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.431708    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8b98810-667c-4ad9-8724-7c64dd6be296-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.509402 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.509324    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerID="77b03a7410c535ea889833902ebc956f7adbe6f834203343968944e1df851b9f" exitCode=0
Sep 19 09:18:21.509402 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.509392    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t" event={"ID":"d7af1339-caf4-4bbf-acf6-84b617dd3416","Type":"ContainerDied","Data":"77b03a7410c535ea889833902ebc956f7adbe6f834203343968944e1df851b9f"}
Sep 19 09:18:21.533083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.533058    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8b98810-667c-4ad9-8724-7c64dd6be296-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.533206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.533127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8b98810-667c-4ad9-8724-7c64dd6be296-tmp\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.533206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.533146    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8b98810-667c-4ad9-8724-7c64dd6be296-manifests\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.533206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.533164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j7g8\" (UniqueName: \"kubernetes.io/projected/c8b98810-667c-4ad9-8724-7c64dd6be296-kube-api-access-2j7g8\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.533206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.533182    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b98810-667c-4ad9-8724-7c64dd6be296-webhook-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.533578 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.533556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8b98810-667c-4ad9-8724-7c64dd6be296-manifests\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.535459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.535440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8b98810-667c-4ad9-8724-7c64dd6be296-tmp\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.535730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.535711    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c8b98810-667c-4ad9-8724-7c64dd6be296-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.535766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.535757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c8b98810-667c-4ad9-8724-7c64dd6be296-webhook-cert\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.549638 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.549615    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j7g8\" (UniqueName: \"kubernetes.io/projected/c8b98810-667c-4ad9-8724-7c64dd6be296-kube-api-access-2j7g8\") pod \"opendatahub-operator-controller-manager-56479975c9-98ss7\" (UID: \"c8b98810-667c-4ad9-8724-7c64dd6be296\") " pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.633406 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.633367    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:21.701500 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.701474    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:21.728752 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.728685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:21.827734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.827634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:21.928769 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.928733    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:21.997702 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:21.997675    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"]
Sep 19 09:18:21.999203 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:18:21.999179    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8b98810_667c_4ad9_8724_7c64dd6be296.slice/crio-4b19149fe4717fdedb231840fda1dd1570d18fa51bf7326a27bdd7810040dd43 WatchSource:0}: Error finding container 4b19149fe4717fdedb231840fda1dd1570d18fa51bf7326a27bdd7810040dd43: Status 404 returned error can't find the container with id 4b19149fe4717fdedb231840fda1dd1570d18fa51bf7326a27bdd7810040dd43
Sep 19 09:18:22.030425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.030368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:22.133513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.133461    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:22.478858 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.478755    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:22.516853 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.516810    2581 generic.go:356] "Generic (PLEG): container finished" podID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerID="f92751547e3e344300208773c51abeed83a70aff20edbf76499fc3f1a75d08df" exitCode=0
Sep 19 09:18:22.517028 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.516877    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t" event={"ID":"d7af1339-caf4-4bbf-acf6-84b617dd3416","Type":"ContainerDied","Data":"f92751547e3e344300208773c51abeed83a70aff20edbf76499fc3f1a75d08df"}
Sep 19 09:18:22.518652 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.518627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7" event={"ID":"c8b98810-667c-4ad9-8724-7c64dd6be296","Type":"ContainerStarted","Data":"4b19149fe4717fdedb231840fda1dd1570d18fa51bf7326a27bdd7810040dd43"}
Sep 19 09:18:22.579984 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.579939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:22.698443 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:22.698398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:25.276189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.276158    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:25.363636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.363602    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-bundle\") pod \"d7af1339-caf4-4bbf-acf6-84b617dd3416\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") "
Sep 19 09:18:25.363636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.363642    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ww2g\" (UniqueName: \"kubernetes.io/projected/d7af1339-caf4-4bbf-acf6-84b617dd3416-kube-api-access-8ww2g\") pod \"d7af1339-caf4-4bbf-acf6-84b617dd3416\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") "
Sep 19 09:18:25.363878 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.363697    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-util\") pod \"d7af1339-caf4-4bbf-acf6-84b617dd3416\" (UID: \"d7af1339-caf4-4bbf-acf6-84b617dd3416\") "
Sep 19 09:18:25.364384 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.364356    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-bundle" (OuterVolumeSpecName: "bundle") pod "d7af1339-caf4-4bbf-acf6-84b617dd3416" (UID: "d7af1339-caf4-4bbf-acf6-84b617dd3416"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:18:25.365907 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.365887    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7af1339-caf4-4bbf-acf6-84b617dd3416-kube-api-access-8ww2g" (OuterVolumeSpecName: "kube-api-access-8ww2g") pod "d7af1339-caf4-4bbf-acf6-84b617dd3416" (UID: "d7af1339-caf4-4bbf-acf6-84b617dd3416"). InnerVolumeSpecName "kube-api-access-8ww2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:18:25.464455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.464421    2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-bundle\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:18:25.464455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.464451    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ww2g\" (UniqueName: \"kubernetes.io/projected/d7af1339-caf4-4bbf-acf6-84b617dd3416-kube-api-access-8ww2g\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:18:25.534266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.534195    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t"
Sep 19 09:18:25.534266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.534202    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08nv7t" event={"ID":"d7af1339-caf4-4bbf-acf6-84b617dd3416","Type":"ContainerDied","Data":"ddc2b763b6f792c857da21650fc6aa8784c043b55b50835b5d8e17200de5dd89"}
Sep 19 09:18:25.534266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:25.534241    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddc2b763b6f792c857da21650fc6aa8784c043b55b50835b5d8e17200de5dd89"
Sep 19 09:18:26.011593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:26.011553    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-util" (OuterVolumeSpecName: "util") pod "d7af1339-caf4-4bbf-acf6-84b617dd3416" (UID: "d7af1339-caf4-4bbf-acf6-84b617dd3416"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:18:26.069036 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:26.068991    2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7af1339-caf4-4bbf-acf6-84b617dd3416-util\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:18:26.540855 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:26.540819    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7" event={"ID":"c8b98810-667c-4ad9-8724-7c64dd6be296","Type":"ContainerStarted","Data":"0910f330a7fe5e97f4b0df5aa57710f1372343c711d8eff78b3144f2cc3bb35b"}
Sep 19 09:18:26.891431 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:26.891397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:27.051491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:27.051385    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:27.546081 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:27.546042    2581 generic.go:356] "Generic (PLEG): container finished" podID="c8b98810-667c-4ad9-8724-7c64dd6be296" containerID="0910f330a7fe5e97f4b0df5aa57710f1372343c711d8eff78b3144f2cc3bb35b" exitCode=0
Sep 19 09:18:27.546505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:27.546122    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7" event={"ID":"c8b98810-667c-4ad9-8724-7c64dd6be296","Type":"ContainerDied","Data":"0910f330a7fe5e97f4b0df5aa57710f1372343c711d8eff78b3144f2cc3bb35b"}
Sep 19 09:18:28.552807 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:28.552765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7" event={"ID":"c8b98810-667c-4ad9-8724-7c64dd6be296","Type":"ContainerStarted","Data":"b8daac4ba5e41c7029c7d1876ea65b9efa4c10f4827c9a373c8469d7ef76ed97"}
Sep 19 09:18:28.553276 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:28.552901    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:28.575799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:28.575734    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7" podStartSLOduration=2.422351591 podStartE2EDuration="7.575714395s" totalImagesPullingTime="4.112895826s" totalInitContainerRuntime="1.040466978s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:28.573386686 +0000 UTC m=+722.260191975" watchObservedRunningTime="2026-09-19 09:18:28.575714395 +0000 UTC m=+722.262519694"
Sep 19 09:18:39.559030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:39.558994    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56479975c9-98ss7"
Sep 19 09:18:40.049998 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:40.049961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:41.228188 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.228119    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:41.564790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.564705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:41.607491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607459    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"]
Sep 19 09:18:41.607734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607721    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerName="extract"
Sep 19 09:18:41.607816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607735    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerName="extract"
Sep 19 09:18:41.607816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607755    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerName="pull"
Sep 19 09:18:41.607816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607761    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerName="pull"
Sep 19 09:18:41.607816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607767    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerName="util"
Sep 19 09:18:41.607816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607772    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerName="util"
Sep 19 09:18:41.607816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.607812    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7af1339-caf4-4bbf-acf6-84b617dd3416" containerName="extract"
Sep 19 09:18:41.615763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.615741    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.619904 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.619885    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 19 09:18:41.620210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.620189    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 19 09:18:41.620283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.620234    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pwbkc\""
Sep 19 09:18:41.628926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.628902    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 09:18:41.629005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.628906    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 09:18:41.646234 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.646209    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"]
Sep 19 09:18:41.692025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.691995    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0a03e76-206c-493a-94ab-980d8926fba5-tmp\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.692202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.692031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbkwz\" (UniqueName: \"kubernetes.io/projected/d0a03e76-206c-493a-94ab-980d8926fba5-kube-api-access-kbkwz\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.692202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.692112    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a03e76-206c-493a-94ab-980d8926fba5-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.718593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.718560    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"]
Sep 19 09:18:41.721837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.721822    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.726828 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.726800    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 19 09:18:41.727530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.727498    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 19 09:18:41.727530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.727518    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 19 09:18:41.727708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.727537    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 19 09:18:41.727708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.727518    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bx8tv\""
Sep 19 09:18:41.727708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.727520    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 19 09:18:41.739025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.738998    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"]
Sep 19 09:18:41.792860 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.792823    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d382c572-f402-4612-9ea9-6a391d2ddb66-manager-config\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.792860 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.792864    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d382c572-f402-4612-9ea9-6a391d2ddb66-metrics-cert\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.793126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.792889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a03e76-206c-493a-94ab-980d8926fba5-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.793126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.792962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d382c572-f402-4612-9ea9-6a391d2ddb66-cert\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.793126 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:18:41.792975    2581 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 19 09:18:41.793126 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:18:41.793021    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0a03e76-206c-493a-94ab-980d8926fba5-tls-certs podName:d0a03e76-206c-493a-94ab-980d8926fba5 nodeName:}" failed. No retries permitted until 2026-09-19 09:18:42.293005246 +0000 UTC m=+735.979810522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d0a03e76-206c-493a-94ab-980d8926fba5-tls-certs") pod "kube-auth-proxy-7d645f99d7-m64x8" (UID: "d0a03e76-206c-493a-94ab-980d8926fba5") : secret "kube-auth-proxy-tls" not found
Sep 19 09:18:41.793126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.793067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc655\" (UniqueName: \"kubernetes.io/projected/d382c572-f402-4612-9ea9-6a391d2ddb66-kube-api-access-vc655\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.793126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.793119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0a03e76-206c-493a-94ab-980d8926fba5-tmp\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.793344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.793145    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbkwz\" (UniqueName: \"kubernetes.io/projected/d0a03e76-206c-493a-94ab-980d8926fba5-kube-api-access-kbkwz\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.795498 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.795475    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0a03e76-206c-493a-94ab-980d8926fba5-tmp\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.807054 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.807023    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbkwz\" (UniqueName: \"kubernetes.io/projected/d0a03e76-206c-493a-94ab-980d8926fba5-kube-api-access-kbkwz\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:41.893532 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.893496    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d382c572-f402-4612-9ea9-6a391d2ddb66-cert\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.893710 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.893549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc655\" (UniqueName: \"kubernetes.io/projected/d382c572-f402-4612-9ea9-6a391d2ddb66-kube-api-access-vc655\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.893710 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.893575    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d382c572-f402-4612-9ea9-6a391d2ddb66-manager-config\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.893710 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.893600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d382c572-f402-4612-9ea9-6a391d2ddb66-metrics-cert\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.894322 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.894301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d382c572-f402-4612-9ea9-6a391d2ddb66-manager-config\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.896138 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.896095    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d382c572-f402-4612-9ea9-6a391d2ddb66-cert\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.896300 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.896281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d382c572-f402-4612-9ea9-6a391d2ddb66-metrics-cert\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:41.910011 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:41.909973    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc655\" (UniqueName: \"kubernetes.io/projected/d382c572-f402-4612-9ea9-6a391d2ddb66-kube-api-access-vc655\") pod \"lws-controller-manager-9b48b7f58-nmwch\" (UID: \"d382c572-f402-4612-9ea9-6a391d2ddb66\") " pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:42.030960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.030924    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:42.165407 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.165319    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:42.296734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.296700    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a03e76-206c-493a-94ab-980d8926fba5-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:42.299286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.299270    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0a03e76-206c-493a-94ab-980d8926fba5-tls-certs\") pod \"kube-auth-proxy-7d645f99d7-m64x8\" (UID: \"d0a03e76-206c-493a-94ab-980d8926fba5\") " pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:42.311392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.311365    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"]
Sep 19 09:18:42.314854 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:18:42.314829    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd382c572_f402_4612_9ea9_6a391d2ddb66.slice/crio-531d2686963db05c4977e9e97893ecd002c7526f14b125b90835ea31d4c61c0d WatchSource:0}: Error finding container 531d2686963db05c4977e9e97893ecd002c7526f14b125b90835ea31d4c61c0d: Status 404 returned error can't find the container with id 531d2686963db05c4977e9e97893ecd002c7526f14b125b90835ea31d4c61c0d
Sep 19 09:18:42.524615 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.524529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"
Sep 19 09:18:42.614505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.614477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch" event={"ID":"d382c572-f402-4612-9ea9-6a391d2ddb66","Type":"ContainerStarted","Data":"531d2686963db05c4977e9e97893ecd002c7526f14b125b90835ea31d4c61c0d"}
Sep 19 09:18:42.821261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:42.821221    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8"]
Sep 19 09:18:42.821628 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:18:42.821607    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a03e76_206c_493a_94ab_980d8926fba5.slice/crio-3b4edadddc6b0200c9215e0b76aa97d6902dd07145c1e013ba2bb45029352eec WatchSource:0}: Error finding container 3b4edadddc6b0200c9215e0b76aa97d6902dd07145c1e013ba2bb45029352eec: Status 404 returned error can't find the container with id 3b4edadddc6b0200c9215e0b76aa97d6902dd07145c1e013ba2bb45029352eec
Sep 19 09:18:43.346497 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:43.346454    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:43.438838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:43.438797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:43.620450 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:43.620371    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8" event={"ID":"d0a03e76-206c-493a-94ab-980d8926fba5","Type":"ContainerStarted","Data":"3b4edadddc6b0200c9215e0b76aa97d6902dd07145c1e013ba2bb45029352eec"}
Sep 19 09:18:44.420832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:44.420764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:44.629195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:44.629159    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch" event={"ID":"d382c572-f402-4612-9ea9-6a391d2ddb66","Type":"ContainerStarted","Data":"134f514945dd239174d1bca0d641aab961f43bfc73021bcdd57d9e7cc5c294e8"}
Sep 19 09:18:44.629391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:44.629330    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:18:44.660700 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:44.660633    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch" podStartSLOduration=2.303720481 podStartE2EDuration="3.660610768s" totalImagesPullingTime="1.356890287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:44.657841467 +0000 UTC m=+738.344646767" watchObservedRunningTime="2026-09-19 09:18:44.660610768 +0000 UTC m=+738.347416064"
Sep 19 09:18:46.640361 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:46.640318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8" event={"ID":"d0a03e76-206c-493a-94ab-980d8926fba5","Type":"ContainerStarted","Data":"74e6a25edc4876eaba7c1fc880857144ad3e9e782d1ae34a5faebc8d23612b85"}
Sep 19 09:18:46.657949 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:46.657897    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d645f99d7-m64x8" podStartSLOduration=2.705048624 podStartE2EDuration="5.657882472s" totalImagesPullingTime="2.952833848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:18:46.656619266 +0000 UTC m=+740.343424568" watchObservedRunningTime="2026-09-19 09:18:46.657882472 +0000 UTC m=+740.344687771"
Sep 19 09:18:47.917921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:47.917881    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:48.058809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:48.058769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:48.597903 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:48.597871    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:48.810938 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:48.810895    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:48.903892 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:48.903853    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:52.241326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:52.241290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:52.303291 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:52.303256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:52.404209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:52.404169    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:52.858572 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:52.858542    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:53.025472 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:53.025441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:53.126729 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:53.126695    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:18:55.636675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:18:55.636639    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-9b48b7f58-nmwch"
Sep 19 09:19:01.635429 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:01.635387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:01.736258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:01.736209    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:02.001415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:02.001294    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:02.149581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:02.149549    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:02.937620 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:02.937580    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:03.039437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:03.039399    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:03.189472 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:03.189382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:53.163187 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:53.163074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:19:53.259928 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:19:53.259888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:06.268711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:06.268657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:07.078580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:07.078550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:10.139654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:10.139612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:10.241655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:10.241616    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:10.437496 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:10.437403    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:12.346668 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:12.346638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:12.498231 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:12.498197    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:14.588526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:14.588482    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:14.738549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:14.738511    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:15.283269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:15.283235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:15.684445 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:15.684411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:16.065337 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:16.065242    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:16.258886 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:16.258844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:20.275404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:20.275365    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:20.380425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:20.380390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:20.685829 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:20.685793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:20.792573 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:20.792538    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:23.103506 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:23.103465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:23.211238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:23.211206    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:25.671650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.671613    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:25.772311 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.772275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:25.849222 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.849189    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"]
Sep 19 09:20:25.852249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.852231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:25.853181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.853153    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:25.854377 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.854359    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 09:20:25.854478 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.854405    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 09:20:25.855369 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.855351    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 19 09:20:25.855423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.855351    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kdmd6\""
Sep 19 09:20:25.855423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.855387    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 19 09:20:25.861719 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.861700    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"]
Sep 19 09:20:25.973015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.972922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94889a2a-5c9f-4ede-9684-975df34d6d91-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:25.973015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.972972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw5nr\" (UniqueName: \"kubernetes.io/projected/94889a2a-5c9f-4ede-9684-975df34d6d91-kube-api-access-gw5nr\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:25.973250 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:25.973044    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94889a2a-5c9f-4ede-9684-975df34d6d91-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.074418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.074383    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94889a2a-5c9f-4ede-9684-975df34d6d91-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.074591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.074476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94889a2a-5c9f-4ede-9684-975df34d6d91-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.074591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.074521    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw5nr\" (UniqueName: \"kubernetes.io/projected/94889a2a-5c9f-4ede-9684-975df34d6d91-kube-api-access-gw5nr\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.074675 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:20:26.074610    2581 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 19 09:20:26.074709 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:20:26.074686    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94889a2a-5c9f-4ede-9684-975df34d6d91-plugin-serving-cert podName:94889a2a-5c9f-4ede-9684-975df34d6d91 nodeName:}" failed. No retries permitted until 2026-09-19 09:20:26.574667553 +0000 UTC m=+840.261472829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/94889a2a-5c9f-4ede-9684-975df34d6d91-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-6td5q" (UID: "94889a2a-5c9f-4ede-9684-975df34d6d91") : secret "plugin-serving-cert" not found
Sep 19 09:20:26.075133 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.075089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94889a2a-5c9f-4ede-9684-975df34d6d91-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.083435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.083414    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw5nr\" (UniqueName: \"kubernetes.io/projected/94889a2a-5c9f-4ede-9684-975df34d6d91-kube-api-access-gw5nr\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.196785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.196746    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:26.577537 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.577494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94889a2a-5c9f-4ede-9684-975df34d6d91-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.580160 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.580133    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94889a2a-5c9f-4ede-9684-975df34d6d91-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6td5q\" (UID: \"94889a2a-5c9f-4ede-9684-975df34d6d91\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:26.763463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.763436    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kdmd6\""
Sep 19 09:20:26.771626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:26.771600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"
Sep 19 09:20:27.046143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:27.046119    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q"]
Sep 19 09:20:27.047806 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:20:27.047782    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94889a2a_5c9f_4ede_9684_975df34d6d91.slice/crio-8753f34f51251ec3fb1139d0f0ffbab212d75a5274c97fb8336ba8f49f320e62 WatchSource:0}: Error finding container 8753f34f51251ec3fb1139d0f0ffbab212d75a5274c97fb8336ba8f49f320e62: Status 404 returned error can't find the container with id 8753f34f51251ec3fb1139d0f0ffbab212d75a5274c97fb8336ba8f49f320e62
Sep 19 09:20:27.067810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:27.067780    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q" event={"ID":"94889a2a-5c9f-4ede-9684-975df34d6d91","Type":"ContainerStarted","Data":"8753f34f51251ec3fb1139d0f0ffbab212d75a5274c97fb8336ba8f49f320e62"}
Sep 19 09:20:32.039891 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:32.039801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:32.095344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:32.095311    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q" event={"ID":"94889a2a-5c9f-4ede-9684-975df34d6d91","Type":"ContainerStarted","Data":"5c0e7f0ea9b0af238d54f770c69c888c98ebaa05164881a2484d6d048a14f330"}
Sep 19 09:20:32.114833 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:32.114780    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6td5q" podStartSLOduration=2.5307242260000002 podStartE2EDuration="7.11476411s" totalImagesPullingTime="4.584039884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:20:32.112028378 +0000 UTC m=+845.798833675" watchObservedRunningTime="2026-09-19 09:20:32.11476411 +0000 UTC m=+845.801569409"
Sep 19 09:20:32.140128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:32.140074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:32.239709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:32.239678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:39.936440 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:39.936404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:40.040908 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:40.040857    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:40.513971 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:40.513929    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:40.614608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:40.614574    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:42.933459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:42.933427    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:43.029416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:43.029377    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:44.735486 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:44.735448    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:44.834227 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:44.834200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:46.115901 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:46.115868    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:46.174880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:46.174844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:52.020218 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:52.020182    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:52.148012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:52.147978    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:20:52.242804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:20:52.242767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:05.519832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:05.519794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:05.849297 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:05.849214    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:21.879284 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:21.879238    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:21.981754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:21.981711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:22.082046 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.082004    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:22.237348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.237253    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:21:22.240391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.240369    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.242785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.242764    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 19 09:21:22.250457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.250436    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:21:22.265584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.265561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2n2m\" (UniqueName: \"kubernetes.io/projected/54b5ae5b-3d7b-46b8-ab65-af70cf445de4-kube-api-access-w2n2m\") pod \"limitador-limitador-84994f7ffc-cxbrk\" (UID: \"54b5ae5b-3d7b-46b8-ab65-af70cf445de4\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.265697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.265603    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/54b5ae5b-3d7b-46b8-ab65-af70cf445de4-config-file\") pod \"limitador-limitador-84994f7ffc-cxbrk\" (UID: \"54b5ae5b-3d7b-46b8-ab65-af70cf445de4\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.284515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.284486    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:21:22.366518 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.366481    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2n2m\" (UniqueName: \"kubernetes.io/projected/54b5ae5b-3d7b-46b8-ab65-af70cf445de4-kube-api-access-w2n2m\") pod \"limitador-limitador-84994f7ffc-cxbrk\" (UID: \"54b5ae5b-3d7b-46b8-ab65-af70cf445de4\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.366724 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.366531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/54b5ae5b-3d7b-46b8-ab65-af70cf445de4-config-file\") pod \"limitador-limitador-84994f7ffc-cxbrk\" (UID: \"54b5ae5b-3d7b-46b8-ab65-af70cf445de4\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.367137 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.367089    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/54b5ae5b-3d7b-46b8-ab65-af70cf445de4-config-file\") pod \"limitador-limitador-84994f7ffc-cxbrk\" (UID: \"54b5ae5b-3d7b-46b8-ab65-af70cf445de4\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.375128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.375072    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2n2m\" (UniqueName: \"kubernetes.io/projected/54b5ae5b-3d7b-46b8-ab65-af70cf445de4-kube-api-access-w2n2m\") pod \"limitador-limitador-84994f7ffc-cxbrk\" (UID: \"54b5ae5b-3d7b-46b8-ab65-af70cf445de4\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.551047 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.550953    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:22.676676 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.676640    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tgsgn"]
Sep 19 09:21:22.680021 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.680004    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn"
Sep 19 09:21:22.682671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.682646    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qs5ww\""
Sep 19 09:21:22.687275 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.687247    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tgsgn"]
Sep 19 09:21:22.714425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.714386    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:22.771395 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.771352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chdzk\" (UniqueName: \"kubernetes.io/projected/77f03cf5-dd13-4d32-add2-5ed98ecd1297-kube-api-access-chdzk\") pod \"authorino-7bcfd8f9cd-tgsgn\" (UID: \"77f03cf5-dd13-4d32-add2-5ed98ecd1297\") " pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn"
Sep 19 09:21:22.793432 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.793383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:22.828963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.826912    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:21:22.833119 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:21:22.833075    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b5ae5b_3d7b_46b8_ab65_af70cf445de4.slice/crio-6a68a8f4aec16a11c4618c61d93cf3ce4647e0488a9b0d3273e3deb6bf263ce0 WatchSource:0}: Error finding container 6a68a8f4aec16a11c4618c61d93cf3ce4647e0488a9b0d3273e3deb6bf263ce0: Status 404 returned error can't find the container with id 6a68a8f4aec16a11c4618c61d93cf3ce4647e0488a9b0d3273e3deb6bf263ce0
Sep 19 09:21:22.872557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.872527    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chdzk\" (UniqueName: \"kubernetes.io/projected/77f03cf5-dd13-4d32-add2-5ed98ecd1297-kube-api-access-chdzk\") pod \"authorino-7bcfd8f9cd-tgsgn\" (UID: \"77f03cf5-dd13-4d32-add2-5ed98ecd1297\") " pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn"
Sep 19 09:21:22.880558 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.880535    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chdzk\" (UniqueName: \"kubernetes.io/projected/77f03cf5-dd13-4d32-add2-5ed98ecd1297-kube-api-access-chdzk\") pod \"authorino-7bcfd8f9cd-tgsgn\" (UID: \"77f03cf5-dd13-4d32-add2-5ed98ecd1297\") " pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn"
Sep 19 09:21:22.990240 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:22.990202    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn"
Sep 19 09:21:23.260612 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:23.260581    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tgsgn"]
Sep 19 09:21:23.262987 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:21:23.262958    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77f03cf5_dd13_4d32_add2_5ed98ecd1297.slice/crio-42f890776112e661b48d0c8b6c084c75c38b145027204c57e7fb0cbc270c778c WatchSource:0}: Error finding container 42f890776112e661b48d0c8b6c084c75c38b145027204c57e7fb0cbc270c778c: Status 404 returned error can't find the container with id 42f890776112e661b48d0c8b6c084c75c38b145027204c57e7fb0cbc270c778c
Sep 19 09:21:23.316202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:23.316152    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn" event={"ID":"77f03cf5-dd13-4d32-add2-5ed98ecd1297","Type":"ContainerStarted","Data":"42f890776112e661b48d0c8b6c084c75c38b145027204c57e7fb0cbc270c778c"}
Sep 19 09:21:23.317320 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:23.317289    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk" event={"ID":"54b5ae5b-3d7b-46b8-ab65-af70cf445de4","Type":"ContainerStarted","Data":"6a68a8f4aec16a11c4618c61d93cf3ce4647e0488a9b0d3273e3deb6bf263ce0"}
Sep 19 09:21:25.329760 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:25.329661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk" event={"ID":"54b5ae5b-3d7b-46b8-ab65-af70cf445de4","Type":"ContainerStarted","Data":"d1ecb86fe3637a76cab3c2c63e2678cbb1a37c1cb66da72d8622943937c4697e"}
Sep 19 09:21:25.330210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:25.329808    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:25.346366 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:25.346305    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk" podStartSLOduration=1.144520597 podStartE2EDuration="3.346285668s" totalImagesPullingTime="2.201765071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:21:25.345612092 +0000 UTC m=+899.032417390" watchObservedRunningTime="2026-09-19 09:21:25.346285668 +0000 UTC m=+899.033090967"
Sep 19 09:21:26.336569 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:26.336483    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn" event={"ID":"77f03cf5-dd13-4d32-add2-5ed98ecd1297","Type":"ContainerStarted","Data":"72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625"}
Sep 19 09:21:26.350232 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:26.350202    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tgsgn"]
Sep 19 09:21:26.360442 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:26.360396    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn" podStartSLOduration=1.651207212 podStartE2EDuration="4.360380138s" totalImagesPullingTime="2.709172926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:21:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:21:26.359979124 +0000 UTC m=+900.046784423" watchObservedRunningTime="2026-09-19 09:21:26.360380138 +0000 UTC m=+900.047185435"
Sep 19 09:21:28.345070 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:28.345028    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn" podUID="77f03cf5-dd13-4d32-add2-5ed98ecd1297" containerName="authorino" containerID="cri-o://72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625" gracePeriod=30
Sep 19 09:21:28.594782 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:28.594754    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn"
Sep 19 09:21:28.620135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:28.620021    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chdzk\" (UniqueName: \"kubernetes.io/projected/77f03cf5-dd13-4d32-add2-5ed98ecd1297-kube-api-access-chdzk\") pod \"77f03cf5-dd13-4d32-add2-5ed98ecd1297\" (UID: \"77f03cf5-dd13-4d32-add2-5ed98ecd1297\") "
Sep 19 09:21:28.622465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:28.622433    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f03cf5-dd13-4d32-add2-5ed98ecd1297-kube-api-access-chdzk" (OuterVolumeSpecName: "kube-api-access-chdzk") pod "77f03cf5-dd13-4d32-add2-5ed98ecd1297" (UID: "77f03cf5-dd13-4d32-add2-5ed98ecd1297"). InnerVolumeSpecName "kube-api-access-chdzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:21:28.720916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:28.720870    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chdzk\" (UniqueName: \"kubernetes.io/projected/77f03cf5-dd13-4d32-add2-5ed98ecd1297-kube-api-access-chdzk\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:21:29.350490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.350454    2581 generic.go:356] "Generic (PLEG): container finished" podID="77f03cf5-dd13-4d32-add2-5ed98ecd1297" containerID="72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625" exitCode=0
Sep 19 09:21:29.350963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.350508    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn"
Sep 19 09:21:29.350963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.350527    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn" event={"ID":"77f03cf5-dd13-4d32-add2-5ed98ecd1297","Type":"ContainerDied","Data":"72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625"}
Sep 19 09:21:29.350963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.350560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-tgsgn" event={"ID":"77f03cf5-dd13-4d32-add2-5ed98ecd1297","Type":"ContainerDied","Data":"42f890776112e661b48d0c8b6c084c75c38b145027204c57e7fb0cbc270c778c"}
Sep 19 09:21:29.350963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.350576    2581 scope.go:117] "RemoveContainer" containerID="72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625"
Sep 19 09:21:29.358778 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.358756    2581 scope.go:117] "RemoveContainer" containerID="72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625"
Sep 19 09:21:29.359045 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:21:29.359028    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625\": container with ID starting with 72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625 not found: ID does not exist" containerID="72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625"
Sep 19 09:21:29.359139 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.359050    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625"} err="failed to get container status \"72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625\": rpc error: code = NotFound desc = could not find container \"72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625\": container with ID starting with 72fe7fe5c57793909dd98a82f40eb5a7df6a9b58c21c27a1d9cbd423d6758625 not found: ID does not exist"
Sep 19 09:21:29.366153 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.366130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tgsgn"]
Sep 19 09:21:29.370166 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:29.370145    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-tgsgn"]
Sep 19 09:21:30.829870 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:30.829838    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77f03cf5-dd13-4d32-add2-5ed98ecd1297" path="/var/lib/kubelet/pods/77f03cf5-dd13-4d32-add2-5ed98ecd1297/volumes"
Sep 19 09:21:36.338079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:36.338043    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"
Sep 19 09:21:38.867281 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:38.867236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:38.963142 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:38.963108    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:39.355608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:39.355576    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:51.784921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:51.784882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:52.030755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.030719    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-gwzx6"]
Sep 19 09:21:52.031014 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.031002    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f03cf5-dd13-4d32-add2-5ed98ecd1297" containerName="authorino"
Sep 19 09:21:52.031056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.031015    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f03cf5-dd13-4d32-add2-5ed98ecd1297" containerName="authorino"
Sep 19 09:21:52.031120 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.031089    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="77f03cf5-dd13-4d32-add2-5ed98ecd1297" containerName="authorino"
Sep 19 09:21:52.033857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.033842    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gwzx6"
Sep 19 09:21:52.036740 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.036680    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qs5ww\""
Sep 19 09:21:52.044089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.044057    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-gwzx6"]
Sep 19 09:21:52.071518 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.071479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:52.107221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.107180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtflv\" (UniqueName: \"kubernetes.io/projected/6c2badf1-a252-49f6-81bd-2230edee69c4-kube-api-access-rtflv\") pod \"authorino-557f876f69-gwzx6\" (UID: \"6c2badf1-a252-49f6-81bd-2230edee69c4\") " pod="kuadrant-system/authorino-557f876f69-gwzx6"
Sep 19 09:21:52.112707 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.112667    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-gwzx6"]
Sep 19 09:21:52.112912 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:21:52.112894    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rtflv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-gwzx6" podUID="6c2badf1-a252-49f6-81bd-2230edee69c4"
Sep 19 09:21:52.208609 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.208568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtflv\" (UniqueName: \"kubernetes.io/projected/6c2badf1-a252-49f6-81bd-2230edee69c4-kube-api-access-rtflv\") pod \"authorino-557f876f69-gwzx6\" (UID: \"6c2badf1-a252-49f6-81bd-2230edee69c4\") " pod="kuadrant-system/authorino-557f876f69-gwzx6"
Sep 19 09:21:52.217040 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.217005    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtflv\" (UniqueName: \"kubernetes.io/projected/6c2badf1-a252-49f6-81bd-2230edee69c4-kube-api-access-rtflv\") pod \"authorino-557f876f69-gwzx6\" (UID: \"6c2badf1-a252-49f6-81bd-2230edee69c4\") " pod="kuadrant-system/authorino-557f876f69-gwzx6"
Sep 19 09:21:52.452185 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.452155    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gwzx6"
Sep 19 09:21:52.459275 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.459252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gwzx6"
Sep 19 09:21:52.611527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.611480    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtflv\" (UniqueName: \"kubernetes.io/projected/6c2badf1-a252-49f6-81bd-2230edee69c4-kube-api-access-rtflv\") pod \"6c2badf1-a252-49f6-81bd-2230edee69c4\" (UID: \"6c2badf1-a252-49f6-81bd-2230edee69c4\") "
Sep 19 09:21:52.613827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.613796    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c2badf1-a252-49f6-81bd-2230edee69c4-kube-api-access-rtflv" (OuterVolumeSpecName: "kube-api-access-rtflv") pod "6c2badf1-a252-49f6-81bd-2230edee69c4" (UID: "6c2badf1-a252-49f6-81bd-2230edee69c4"). InnerVolumeSpecName "kube-api-access-rtflv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:21:52.712738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:52.712640    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtflv\" (UniqueName: \"kubernetes.io/projected/6c2badf1-a252-49f6-81bd-2230edee69c4-kube-api-access-rtflv\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:21:53.456486 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:53.456454    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gwzx6"
Sep 19 09:21:53.488279 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:53.488244    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-gwzx6"]
Sep 19 09:21:53.495917 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:53.495875    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-gwzx6"]
Sep 19 09:21:54.830456 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:54.830420    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c2badf1-a252-49f6-81bd-2230edee69c4" path="/var/lib/kubelet/pods/6c2badf1-a252-49f6-81bd-2230edee69c4/volumes"
Sep 19 09:21:58.219977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:58.219939    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:58.298969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:58.298936    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:58.357229 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:58.357185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:58.470299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:58.470220    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:21:58.582804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:21:58.582767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:01.731526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:01.731491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:01.838589 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:01.838548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:32.529923 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:32.529883    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:32.623954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:32.623913    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:34.008043 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:34.008014    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:34.109318 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:34.109265    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:34.279066 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:34.278982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:34.412747 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:34.412708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:34.633458 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:34.633424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:34.737929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:34.737890    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:34.845164 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:34.845128    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:22:44.689560 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:44.689513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:22:44.784054 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:22:44.784018    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:02.052342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:02.052260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:02.107436 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:02.107404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:02.410299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:02.410271    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:02.976344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:02.976310    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:23:05.805039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:05.805007    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:06.464960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:06.464927    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:23:19.812842 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:19.812807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:20.870694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:20.870658    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:23:28.362956 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.362925    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"]
Sep 19 09:23:28.366171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.366151    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.368854 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.368833    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-942q6\""
Sep 19 09:23:28.369719 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.369702    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 09:23:28.369808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.369769    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 19 09:23:28.369867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.369829    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 09:23:28.381020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.380998    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"]
Sep 19 09:23:28.406627 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.406594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:28.451349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.451316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.451517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.451358    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9e7a2b5-8640-421a-b79e-451a01510c01-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.451517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.451384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pppn\" (UniqueName: \"kubernetes.io/projected/b9e7a2b5-8640-421a-b79e-451a01510c01-kube-api-access-9pppn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.451517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.451483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.451650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.451547    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.451650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.451576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552427    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9e7a2b5-8640-421a-b79e-451a01510c01-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pppn\" (UniqueName: \"kubernetes.io/projected/b9e7a2b5-8640-421a-b79e-451a01510c01-kube-api-access-9pppn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552890 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552945 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.552993 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.552971    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.554935 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.554916    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9e7a2b5-8640-421a-b79e-451a01510c01-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.555230 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.555212    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9e7a2b5-8640-421a-b79e-451a01510c01-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.561708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.561681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pppn\" (UniqueName: \"kubernetes.io/projected/b9e7a2b5-8640-421a-b79e-451a01510c01-kube-api-access-9pppn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-bjpd5\" (UID: \"b9e7a2b5-8640-421a-b79e-451a01510c01\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.675983 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.675942    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:28.950342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.950259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"]
Sep 19 09:23:28.953488 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:23:28.953459    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e7a2b5_8640_421a_b79e_451a01510c01.slice/crio-43c2ad5592760ce6b9eaaace6f6df1e3ada25d02661247d428ef7fba9a463ab9 WatchSource:0}: Error finding container 43c2ad5592760ce6b9eaaace6f6df1e3ada25d02661247d428ef7fba9a463ab9: Status 404 returned error can't find the container with id 43c2ad5592760ce6b9eaaace6f6df1e3ada25d02661247d428ef7fba9a463ab9
Sep 19 09:23:28.955203 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:28.955184    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 09:23:29.568851 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:29.568818    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:23:29.879885 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:29.879849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5" event={"ID":"b9e7a2b5-8640-421a-b79e-451a01510c01","Type":"ContainerStarted","Data":"43c2ad5592760ce6b9eaaace6f6df1e3ada25d02661247d428ef7fba9a463ab9"}
Sep 19 09:23:30.886174 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:30.886130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5" event={"ID":"b9e7a2b5-8640-421a-b79e-451a01510c01","Type":"ContainerStarted","Data":"0d7890571edb72efe571bdd921c870d84f97bf61871dc62b18422501ecc6d57b"}
Sep 19 09:23:30.904550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:30.904502    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5" podStartSLOduration=1.903815745 podStartE2EDuration="2.904488032s" totalImagesPullingTime="1.000672287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:23:30.904233333 +0000 UTC m=+1024.591038634" watchObservedRunningTime="2026-09-19 09:23:30.904488032 +0000 UTC m=+1024.591293329"
Sep 19 09:23:38.676701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:38.676654    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:38.677301 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:38.676726    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:38.689575 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:38.689547    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:38.930514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:38.930419    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-bjpd5"
Sep 19 09:23:39.004373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:39.004337    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:40.565693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:40.565662    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:23:42.664338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:42.664305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:23:59.939965 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:23:59.939928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:24:00.041552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.041518    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:24:00.279290 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.279210    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"]
Sep 19 09:24:00.285303 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.285283    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.288383 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.288362    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 19 09:24:00.296853 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.296831    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"]
Sep 19 09:24:00.308181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.308150    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.308351 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.308224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.308351 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.308280    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a60185-6e3e-434c-9001-f6e09e21c7f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.308351 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.308316    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.308508 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.308366    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxdpk\" (UniqueName: \"kubernetes.io/projected/91a60185-6e3e-434c-9001-f6e09e21c7f2-kube-api-access-gxdpk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.308508 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.308389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.323290 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.323253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:24:00.409044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a60185-6e3e-434c-9001-f6e09e21c7f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxdpk\" (UniqueName: \"kubernetes.io/projected/91a60185-6e3e-434c-9001-f6e09e21c7f2-kube-api-access-gxdpk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409435    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409571 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409519    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.409617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.409601    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.411590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.411568    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91a60185-6e3e-434c-9001-f6e09e21c7f2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.411876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.411856    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91a60185-6e3e-434c-9001-f6e09e21c7f2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.417125 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.417086    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxdpk\" (UniqueName: \"kubernetes.io/projected/91a60185-6e3e-434c-9001-f6e09e21c7f2-kube-api-access-gxdpk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bmwg5\" (UID: \"91a60185-6e3e-434c-9001-f6e09e21c7f2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.595838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.595749    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:00.872009 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:00.871988    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"]
Sep 19 09:24:00.874762 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:24:00.874736    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a60185_6e3e_434c_9001_f6e09e21c7f2.slice/crio-57a4396be4ce6d32f7b3f3d88e98e5390ee640182f79f3c2908ffb84d9470a00 WatchSource:0}: Error finding container 57a4396be4ce6d32f7b3f3d88e98e5390ee640182f79f3c2908ffb84d9470a00: Status 404 returned error can't find the container with id 57a4396be4ce6d32f7b3f3d88e98e5390ee640182f79f3c2908ffb84d9470a00
Sep 19 09:24:01.023082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:01.023050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5" event={"ID":"91a60185-6e3e-434c-9001-f6e09e21c7f2","Type":"ContainerStarted","Data":"57a4396be4ce6d32f7b3f3d88e98e5390ee640182f79f3c2908ffb84d9470a00"}
Sep 19 09:24:02.029006 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:02.028973    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5" event={"ID":"91a60185-6e3e-434c-9001-f6e09e21c7f2","Type":"ContainerStarted","Data":"ba54a02d6b794ceb010bf49a54403bd4aa662e1011b1c0f1bfd702ad13129f37"}
Sep 19 09:24:02.048875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:02.048823    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5" podStartSLOduration=1.876687214 podStartE2EDuration="2.048805614s" totalImagesPullingTime="172.1184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:24:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:24:02.0480515 +0000 UTC m=+1055.734856798" watchObservedRunningTime="2026-09-19 09:24:02.048805614 +0000 UTC m=+1055.735610906"
Sep 19 09:24:02.078655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:02.078619    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:24:10.596581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:10.596535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:10.597141 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:10.596621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:10.609052 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:10.609027    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:11.079001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:11.078973    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bmwg5"
Sep 19 09:24:14.971079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:14.971039    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"]
Sep 19 09:24:14.974759 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:14.974737    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:14.977009 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:14.976988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 19 09:24:14.986214 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:14.986187    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"]
Sep 19 09:24:15.013848 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.013816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:24:15.030717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.030687    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.030839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.030728    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.030839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.030756    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqdtx\" (UniqueName: \"kubernetes.io/projected/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-kube-api-access-jqdtx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.030932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.030833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.030932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.030869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.030932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.030887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.131535 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.131493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.131535 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.131538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.131788 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.131578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.131788 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.131599    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.131788 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.131625    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqdtx\" (UniqueName: \"kubernetes.io/projected/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-kube-api-access-jqdtx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.131942 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.131843    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.132012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.131988    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.132069 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.132049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.132200 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.132182    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.134039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.134011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.134419 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.134398    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.140093 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.140069    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqdtx\" (UniqueName: \"kubernetes.io/projected/ba6c19fa-1d9f-4e21-9144-dd12f22b25f9-kube-api-access-jqdtx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-55bsw\" (UID: \"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.285325 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.285237    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:15.559161 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:15.559135    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"]
Sep 19 09:24:15.561845 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:24:15.561813    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba6c19fa_1d9f_4e21_9144_dd12f22b25f9.slice/crio-8b36a5eefe628b3bcde708b97b4b144b7d8f13208312b5e364c85e721b6a48dd WatchSource:0}: Error finding container 8b36a5eefe628b3bcde708b97b4b144b7d8f13208312b5e364c85e721b6a48dd: Status 404 returned error can't find the container with id 8b36a5eefe628b3bcde708b97b4b144b7d8f13208312b5e364c85e721b6a48dd
Sep 19 09:24:16.092458 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:16.092414    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw" event={"ID":"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9","Type":"ContainerStarted","Data":"b552c6ebc75567793ffaa8390f5866851b2f707fe9076813175474f8727656be"}
Sep 19 09:24:16.092458 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:16.092460    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw" event={"ID":"ba6c19fa-1d9f-4e21-9144-dd12f22b25f9","Type":"ContainerStarted","Data":"8b36a5eefe628b3bcde708b97b4b144b7d8f13208312b5e364c85e721b6a48dd"}
Sep 19 09:24:16.111531 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:16.111483    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw" podStartSLOduration=1.938105604 podStartE2EDuration="2.111467258s" totalImagesPullingTime="173.361654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:24:16.109516525 +0000 UTC m=+1069.796321845" watchObservedRunningTime="2026-09-19 09:24:16.111467258 +0000 UTC m=+1069.798272556"
Sep 19 09:24:17.301133 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:17.301083    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:24:25.285807 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:25.285775    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:25.286215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:25.285829    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:25.298136 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:25.298092    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:26.147243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:26.147213    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-55bsw"
Sep 19 09:24:41.173821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:24:41.173786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:29.096194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:29.096151    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:32.201518 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:32.201479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:32.300072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:32.300034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:33.274731 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:33.274686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:33.370561 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:33.370530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:51.641672 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:51.641586    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:51.744300 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:51.744253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:51.855172 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:51.855137    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:52.138830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:52.138794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:52.239717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:52.239679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:52.334507 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:52.334474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:52.591804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:52.591708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:52.761433 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:52.761403    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:53.027430 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.027391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:53.117952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.117910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:53.217639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.217605    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:53.462295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.462261    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:53.620315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.620285    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-7d2b28"]
Sep 19 09:25:53.623377 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.623358    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:25:53.625810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.625787    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q5mhq\""
Sep 19 09:25:53.630876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.630444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-7d2b28"]
Sep 19 09:25:53.741403 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.741309    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr67t\" (UniqueName: \"kubernetes.io/projected/8686656a-8e96-40c8-bf59-e82a2fe69b3b-kube-api-access-wr67t\") pod \"test-curl-7857-7d2b28\" (UID: \"8686656a-8e96-40c8-bf59-e82a2fe69b3b\") " pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:25:53.842153 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.842085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr67t\" (UniqueName: \"kubernetes.io/projected/8686656a-8e96-40c8-bf59-e82a2fe69b3b-kube-api-access-wr67t\") pod \"test-curl-7857-7d2b28\" (UID: \"8686656a-8e96-40c8-bf59-e82a2fe69b3b\") " pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:25:53.851696 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.851660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr67t\" (UniqueName: \"kubernetes.io/projected/8686656a-8e96-40c8-bf59-e82a2fe69b3b-kube-api-access-wr67t\") pod \"test-curl-7857-7d2b28\" (UID: \"8686656a-8e96-40c8-bf59-e82a2fe69b3b\") " pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:25:53.933713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:53.933671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:25:54.203291 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:54.203267    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-7d2b28"]
Sep 19 09:25:54.205875 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:25:54.205843    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8686656a_8e96_40c8_bf59_e82a2fe69b3b.slice/crio-e069790c8f35c15e057f75f3f083a7a7b66f7fcb06ec56bc1f856e65db6a213c WatchSource:0}: Error finding container e069790c8f35c15e057f75f3f083a7a7b66f7fcb06ec56bc1f856e65db6a213c: Status 404 returned error can't find the container with id e069790c8f35c15e057f75f3f083a7a7b66f7fcb06ec56bc1f856e65db6a213c
Sep 19 09:25:54.546059 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:54.545966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-7d2b28" event={"ID":"8686656a-8e96-40c8-bf59-e82a2fe69b3b","Type":"ContainerStarted","Data":"e069790c8f35c15e057f75f3f083a7a7b66f7fcb06ec56bc1f856e65db6a213c"}
Sep 19 09:25:54.782838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:54.782807    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:54.902113 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:54.902056    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:55.002256 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:55.002216    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:55.301909 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:55.301823    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:25:55.745022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:55.744979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:55.843814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:55.843765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:56.558621 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:56.558531    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-7d2b28" event={"ID":"8686656a-8e96-40c8-bf59-e82a2fe69b3b","Type":"ContainerStarted","Data":"487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b"}
Sep 19 09:25:56.574503 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:56.574369    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7857-7d2b28" podStartSLOduration=1.6339672859999999 podStartE2EDuration="3.574348475s" totalImagesPullingTime="1.940381189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:25:56.573531123 +0000 UTC m=+1170.260336421" watchObservedRunningTime="2026-09-19 09:25:56.574348475 +0000 UTC m=+1170.261153774"
Sep 19 09:25:56.796483 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:56.796422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:56.893520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:56.893483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:57.033575 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.033537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:57.228003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.227914    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-7d2b28"]
Sep 19 09:25:57.234276 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.234248    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-7d2b28"]
Sep 19 09:25:57.357228 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.357199    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7857-52a3a7"]
Sep 19 09:25:57.357506 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.357494    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8686656a-8e96-40c8-bf59-e82a2fe69b3b" containerName="test-curl-7857-7d2b28"
Sep 19 09:25:57.357550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.357509    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8686656a-8e96-40c8-bf59-e82a2fe69b3b" containerName="test-curl-7857-7d2b28"
Sep 19 09:25:57.357588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.357571    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8686656a-8e96-40c8-bf59-e82a2fe69b3b" containerName="test-curl-7857-7d2b28"
Sep 19 09:25:57.360528 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.360514    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:25:57.369881 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.369854    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-52a3a7"]
Sep 19 09:25:57.371869 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.371846    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlj5x\" (UniqueName: \"kubernetes.io/projected/4e8937cf-341f-4c3b-ac04-015640744552-kube-api-access-vlj5x\") pod \"test-curl-7857-52a3a7\" (UID: \"4e8937cf-341f-4c3b-ac04-015640744552\") " pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:25:57.390594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.390531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:57.472383 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.472353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlj5x\" (UniqueName: \"kubernetes.io/projected/4e8937cf-341f-4c3b-ac04-015640744552-kube-api-access-vlj5x\") pod \"test-curl-7857-52a3a7\" (UID: \"4e8937cf-341f-4c3b-ac04-015640744552\") " pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:25:57.481457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.481397    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlj5x\" (UniqueName: \"kubernetes.io/projected/4e8937cf-341f-4c3b-ac04-015640744552-kube-api-access-vlj5x\") pod \"test-curl-7857-52a3a7\" (UID: \"4e8937cf-341f-4c3b-ac04-015640744552\") " pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:25:57.489414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.489382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:57.564332 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:25:57.564302    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7857-7d2b28" container="test-curl-7857-7d2b28"
Sep 19 09:25:57.669711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.669667    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:25:57.776814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.776706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:57.897804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.897774    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:57.948228 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:57.948203    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7857-52a3a7"]
Sep 19 09:25:57.950945 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:25:57.950919    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e8937cf_341f_4c3b_ac04_015640744552.slice/crio-4f468866f50694052fb01e36326753eb6c61fbff8068670e3bc137957d8ceb9c WatchSource:0}: Error finding container 4f468866f50694052fb01e36326753eb6c61fbff8068670e3bc137957d8ceb9c: Status 404 returned error can't find the container with id 4f468866f50694052fb01e36326753eb6c61fbff8068670e3bc137957d8ceb9c
Sep 19 09:25:58.176290 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:58.176259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:58.456036 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:58.455946    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:58.569966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:58.569927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-52a3a7" event={"ID":"4e8937cf-341f-4c3b-ac04-015640744552","Type":"ContainerStarted","Data":"8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350"}
Sep 19 09:25:58.570408 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:58.569975    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7857-52a3a7" event={"ID":"4e8937cf-341f-4c3b-ac04-015640744552","Type":"ContainerStarted","Data":"4f468866f50694052fb01e36326753eb6c61fbff8068670e3bc137957d8ceb9c"}
Sep 19 09:25:58.570408 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:58.570056    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7857-7d2b28" podUID="8686656a-8e96-40c8-bf59-e82a2fe69b3b" containerName="test-curl-7857-7d2b28" containerID="cri-o://487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b" gracePeriod=2
Sep 19 09:25:58.609950 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:58.609902    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7857-52a3a7" podStartSLOduration=1.609887474 podStartE2EDuration="1.609887474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:25:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:25:58.607781565 +0000 UTC m=+1172.294586863" watchObservedRunningTime="2026-09-19 09:25:58.609887474 +0000 UTC m=+1172.296692772"
Sep 19 09:25:58.616988 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:25:58.616952    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7857-7d2b28\" is forbidden: User \"system:node:ip-10-0-131-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-218.ec2.internal' and this object" podUID="8686656a-8e96-40c8-bf59-e82a2fe69b3b" pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:25:58.622146 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:58.622117    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:59.353550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.353513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:59.557050 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.556951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:25:59.636850 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.636815    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7857-52a3a7"]
Sep 19 09:25:59.656470 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.656434    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7857-52a3a7"]
Sep 19 09:25:59.708409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.708375    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"]
Sep 19 09:25:59.708691 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.708679    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e8937cf-341f-4c3b-ac04-015640744552" containerName="test-curl-7857-52a3a7"
Sep 19 09:25:59.708733 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.708693    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e8937cf-341f-4c3b-ac04-015640744552" containerName="test-curl-7857-52a3a7"
Sep 19 09:25:59.708766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.708747    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e8937cf-341f-4c3b-ac04-015640744552" containerName="test-curl-7857-52a3a7"
Sep 19 09:25:59.711751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.711734    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:25:59.717581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.717556    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-acc640-dockercfg-txjjp\""
Sep 19 09:25:59.717722 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.717620    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-acc640\""
Sep 19 09:25:59.741656 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.741623    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"]
Sep 19 09:25:59.793600 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.793554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:25:59.793793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.793638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6vx6\" (UniqueName: \"kubernetes.io/projected/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-kube-api-access-t6vx6\") pod \"payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:25:59.894654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.894620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6vx6\" (UniqueName: \"kubernetes.io/projected/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-kube-api-access-t6vx6\") pod \"payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:25:59.894814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.894670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:25:59.895357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.895339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:25:59.918987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:25:59.918948    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6vx6\" (UniqueName: \"kubernetes.io/projected/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-kube-api-access-t6vx6\") pod \"payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:26:00.021921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.021873    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:26:00.184285 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.184198    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:00.301920 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.301880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:00.308379 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.308357    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"]
Sep 19 09:26:00.310630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:00.310604    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9b9c40b_89ee_452e_98d4_0a5f7a6d67ad.slice/crio-b6a2a96be725b94bfd20e8f7690801bcc7748c56af56283d23b46e5e2e675dc2 WatchSource:0}: Error finding container b6a2a96be725b94bfd20e8f7690801bcc7748c56af56283d23b46e5e2e675dc2: Status 404 returned error can't find the container with id b6a2a96be725b94bfd20e8f7690801bcc7748c56af56283d23b46e5e2e675dc2
Sep 19 09:26:00.564910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.564814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:00.581816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.581782    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj" event={"ID":"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad","Type":"ContainerStarted","Data":"b6a2a96be725b94bfd20e8f7690801bcc7748c56af56283d23b46e5e2e675dc2"}
Sep 19 09:26:00.581940 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.581868    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7857-52a3a7" podUID="4e8937cf-341f-4c3b-ac04-015640744552" containerName="test-curl-7857-52a3a7" containerID="cri-o://8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350" gracePeriod=2
Sep 19 09:26:00.584769 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:00.584742    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7857-52a3a7\" is forbidden: User \"system:node:ip-10-0-131-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-218.ec2.internal' and this object" podUID="4e8937cf-341f-4c3b-ac04-015640744552" pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:26:00.739920 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.739895    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:26:00.742504 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:00.742480    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7857-52a3a7\" is forbidden: User \"system:node:ip-10-0-131-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-218.ec2.internal' and this object" podUID="4e8937cf-341f-4c3b-ac04-015640744552" pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:26:00.772416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.772371    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:00.802431 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.802398    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr67t\" (UniqueName: \"kubernetes.io/projected/8686656a-8e96-40c8-bf59-e82a2fe69b3b-kube-api-access-wr67t\") pod \"8686656a-8e96-40c8-bf59-e82a2fe69b3b\" (UID: \"8686656a-8e96-40c8-bf59-e82a2fe69b3b\") "
Sep 19 09:26:00.804856 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.804825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8686656a-8e96-40c8-bf59-e82a2fe69b3b-kube-api-access-wr67t" (OuterVolumeSpecName: "kube-api-access-wr67t") pod "8686656a-8e96-40c8-bf59-e82a2fe69b3b" (UID: "8686656a-8e96-40c8-bf59-e82a2fe69b3b"). InnerVolumeSpecName "kube-api-access-wr67t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:26:00.833122 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.833021    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8686656a-8e96-40c8-bf59-e82a2fe69b3b" path="/var/lib/kubelet/pods/8686656a-8e96-40c8-bf59-e82a2fe69b3b/volumes"
Sep 19 09:26:00.903302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:00.903260    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr67t\" (UniqueName: \"kubernetes.io/projected/8686656a-8e96-40c8-bf59-e82a2fe69b3b-kube-api-access-wr67t\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:01.179834 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.179787    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:01.589115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.589062    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj" event={"ID":"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad","Type":"ContainerStarted","Data":"f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30"}
Sep 19 09:26:01.589304 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.589232    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:26:01.590663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.590625    2581 generic.go:356] "Generic (PLEG): container finished" podID="8686656a-8e96-40c8-bf59-e82a2fe69b3b" containerID="487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b" exitCode=137
Sep 19 09:26:01.590663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.590659    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-7d2b28"
Sep 19 09:26:01.590846 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.590689    2581 scope.go:117] "RemoveContainer" containerID="487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b"
Sep 19 09:26:01.599209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.599190    2581 scope.go:117] "RemoveContainer" containerID="487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b"
Sep 19 09:26:01.599550 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:01.599532    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b\": container with ID starting with 487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b not found: ID does not exist" containerID="487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b"
Sep 19 09:26:01.599611 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.599558    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b"} err="failed to get container status \"487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b\": rpc error: code = NotFound desc = could not find container \"487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b\": container with ID starting with 487fb50053bfe66380d07d133b1f4aa39dab5266cb01f52e8853eafaaee8534b not found: ID does not exist"
Sep 19 09:26:01.608931 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:01.608880    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj" podStartSLOduration=1.4223815690000001 podStartE2EDuration="2.608864404s" totalImagesPullingTime="1.186482835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:01.606979407 +0000 UTC m=+1175.293784743" watchObservedRunningTime="2026-09-19 09:26:01.608864404 +0000 UTC m=+1175.295669723"
Sep 19 09:26:02.140703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.140674    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:02.213211 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.213173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:02.373041 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.372997    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:02.472426 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.472336    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:02.612190 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.612160    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:02.748809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.748783    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:26:02.790600 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.790567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:02.823140 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.823084    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlj5x\" (UniqueName: \"kubernetes.io/projected/4e8937cf-341f-4c3b-ac04-015640744552-kube-api-access-vlj5x\") pod \"4e8937cf-341f-4c3b-ac04-015640744552\" (UID: \"4e8937cf-341f-4c3b-ac04-015640744552\") "
Sep 19 09:26:02.825344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.825315    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e8937cf-341f-4c3b-ac04-015640744552-kube-api-access-vlj5x" (OuterVolumeSpecName: "kube-api-access-vlj5x") pod "4e8937cf-341f-4c3b-ac04-015640744552" (UID: "4e8937cf-341f-4c3b-ac04-015640744552"). InnerVolumeSpecName "kube-api-access-vlj5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:26:02.830310 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.830282    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e8937cf-341f-4c3b-ac04-015640744552" path="/var/lib/kubelet/pods/4e8937cf-341f-4c3b-ac04-015640744552/volumes"
Sep 19 09:26:02.907671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.907636    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:02.923909 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.923881    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlj5x\" (UniqueName: \"kubernetes.io/projected/4e8937cf-341f-4c3b-ac04-015640744552-kube-api-access-vlj5x\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:02.947856 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.947826    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"]
Sep 19 09:26:02.951264 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.951242    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:02.958048 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.958016    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 09:26:02.958187 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.958073    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b272f719-openshift-default-dockercfg-xnvf6\""
Sep 19 09:26:02.978911 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:02.978882    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"]
Sep 19 09:26:03.007985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.007908    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"]
Sep 19 09:26:03.011204 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.011187    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.017029 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.017001    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-9dfb0f-dockercfg-8tchb\""
Sep 19 09:26:03.017526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.017120    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-9dfb0f\""
Sep 19 09:26:03.024491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024465    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-podinfo\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024502    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-token\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024530    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx47n\" (UniqueName: \"kubernetes.io/projected/2df89df4-5099-4763-8b9b-beef4f2e101d-kube-api-access-wx47n\") pod \"payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.024630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024590    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istiod-ca-cert\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-socket\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024677    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-certs\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024729    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-envoy\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024833 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df89df4-5099-4763-8b9b-beef4f2e101d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.024833 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-data\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024900 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024831    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmpp6\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-kube-api-access-dmpp6\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.024900 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.024861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-credential-socket\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.029198 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.029169    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"]
Sep 19 09:26:03.072221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.072183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:03.125368 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx47n\" (UniqueName: \"kubernetes.io/projected/2df89df4-5099-4763-8b9b-beef4f2e101d-kube-api-access-wx47n\") pod \"payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.125511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125375    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istiod-ca-cert\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.125511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-socket\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.125511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-certs\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.125714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125509    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-envoy\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.125714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125540    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df89df4-5099-4763-8b9b-beef4f2e101d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.125714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-data\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.125714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmpp6\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-kube-api-access-dmpp6\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.125714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-credential-socket\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126019 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125719    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-podinfo\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126019 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125751    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-token\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126019 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125871    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-socket\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126019 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.125954    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-data\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.126138    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-credential-socket\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.126196    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-certs\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.126269    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istiod-ca-cert\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.126521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.126497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df89df4-5099-4763-8b9b-beef4f2e101d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.128343 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.128312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-envoy\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.128741 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.128720    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-podinfo\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.167974 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.167932    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-token\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.168326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.168047    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmpp6\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-kube-api-access-dmpp6\") pod \"e2e-ait-b272f719-openshift-default-cc499f476-kpm2z\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") " pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.171335 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.171317    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx47n\" (UniqueName: \"kubernetes.io/projected/2df89df4-5099-4763-8b9b-beef4f2e101d-kube-api-access-wx47n\") pod \"payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.261499 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.261397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:03.321817 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.321716    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:03.544527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.544498    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"]
Sep 19 09:26:03.546444 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:03.546418    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0cc9ea9_a2f6_45c0_bc46_67b89752ea9b.slice/crio-f2f8229d6d21dd004d677bebbffebda87d2af0f1373fa3b056d1b4bc68120f70 WatchSource:0}: Error finding container f2f8229d6d21dd004d677bebbffebda87d2af0f1373fa3b056d1b4bc68120f70: Status 404 returned error can't find the container with id f2f8229d6d21dd004d677bebbffebda87d2af0f1373fa3b056d1b4bc68120f70
Sep 19 09:26:03.549662 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.549153    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:03.603976 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.603913    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"]
Sep 19 09:26:03.604862 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.604831    2581 generic.go:356] "Generic (PLEG): container finished" podID="4e8937cf-341f-4c3b-ac04-015640744552" containerID="8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350" exitCode=137
Sep 19 09:26:03.604977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.604886    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7857-52a3a7"
Sep 19 09:26:03.604977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.604936    2581 scope.go:117] "RemoveContainer" containerID="8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350"
Sep 19 09:26:03.606612 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.606583    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z" event={"ID":"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b","Type":"ContainerStarted","Data":"f2f8229d6d21dd004d677bebbffebda87d2af0f1373fa3b056d1b4bc68120f70"}
Sep 19 09:26:03.606853 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:03.606829    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df89df4_5099_4763_8b9b_beef4f2e101d.slice/crio-c2d92653feba0a1afec2d2595e0322c5263b02e1e59522f5b25dfea4a4936907 WatchSource:0}: Error finding container c2d92653feba0a1afec2d2595e0322c5263b02e1e59522f5b25dfea4a4936907: Status 404 returned error can't find the container with id c2d92653feba0a1afec2d2595e0322c5263b02e1e59522f5b25dfea4a4936907
Sep 19 09:26:03.618813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.618793    2581 scope.go:117] "RemoveContainer" containerID="8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350"
Sep 19 09:26:03.619148 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:03.619127    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350\": container with ID starting with 8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350 not found: ID does not exist" containerID="8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350"
Sep 19 09:26:03.619202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.619158    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350"} err="failed to get container status \"8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350\": rpc error: code = NotFound desc = could not find container \"8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350\": container with ID starting with 8dd4ba6d1fe9d863d69f7c64d8f6f08333322fd9c13eb6c4100b3e4f6965b350 not found: ID does not exist"
Sep 19 09:26:03.742779 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:03.742745    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:04.015793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:04.015756    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:04.106543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:04.106504    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:04.615305 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:04.614996    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8" event={"ID":"2df89df4-5099-4763-8b9b-beef4f2e101d","Type":"ContainerStarted","Data":"475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8"}
Sep 19 09:26:04.615305 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:04.615061    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8" event={"ID":"2df89df4-5099-4763-8b9b-beef4f2e101d","Type":"ContainerStarted","Data":"c2d92653feba0a1afec2d2595e0322c5263b02e1e59522f5b25dfea4a4936907"}
Sep 19 09:26:04.615305 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:04.615140    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:04.647494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:04.647442    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8" podStartSLOduration=2.647422918 podStartE2EDuration="2.647422918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:04.644172095 +0000 UTC m=+1178.330977393" watchObservedRunningTime="2026-09-19 09:26:04.647422918 +0000 UTC m=+1178.334228218"
Sep 19 09:26:04.993796 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:04.993733    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:05.881062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:05.881022    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:26:05.881447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:05.881131    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:26:05.881447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:05.881178    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:26:05.954253 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:05.954221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:06.628650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:06.628616    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z" event={"ID":"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b","Type":"ContainerStarted","Data":"63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9"}
Sep 19 09:26:06.652476 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:06.652420    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z" podStartSLOduration=2.3203732329999998 podStartE2EDuration="4.65240104s" totalImagesPullingTime="2.332027807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:06.650977538 +0000 UTC m=+1180.337782837" watchObservedRunningTime="2026-09-19 09:26:06.65240104 +0000 UTC m=+1180.339206339"
Sep 19 09:26:06.916032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:06.915956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:07.261688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:07.261604    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:07.266760 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:07.266737    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:07.633812 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:07.633783    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:07.634672 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:07.634653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:07.845890 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:07.845851    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:07.947959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:07.947866    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:08.061865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.061832    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:08.154452 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.154417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:08.555675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.555643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:08.656284 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.656251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:08.752577 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.752527    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"]
Sep 19 09:26:08.756071 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.756053    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:08.758506 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.758482    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0ce808-dockercfg-zkttz\""
Sep 19 09:26:08.759000 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.758981    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0ce808\""
Sep 19 09:26:08.767455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.767433    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"]
Sep 19 09:26:08.875861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.875780    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8730d4c-92c0-4fad-becc-2a066444db6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:08.875861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.875835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klg28\" (UniqueName: \"kubernetes.io/projected/f8730d4c-92c0-4fad-becc-2a066444db6e-kube-api-access-klg28\") pod \"payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:08.977205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.977163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klg28\" (UniqueName: \"kubernetes.io/projected/f8730d4c-92c0-4fad-becc-2a066444db6e-kube-api-access-klg28\") pod \"payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:08.977622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.977242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8730d4c-92c0-4fad-becc-2a066444db6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:08.977872 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.977853    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8730d4c-92c0-4fad-becc-2a066444db6e-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:08.986431 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:08.986400    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klg28\" (UniqueName: \"kubernetes.io/projected/f8730d4c-92c0-4fad-becc-2a066444db6e-kube-api-access-klg28\") pod \"payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:09.067387 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.067333    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:09.229442 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.229403    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:09.359832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.359799    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"]
Sep 19 09:26:09.363057 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:09.363027    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8730d4c_92c0_4fad_becc_2a066444db6e.slice/crio-c23fd44051ca0b4dd82c043678040844f8c67b515adabb9f811f3974952ef1ec WatchSource:0}: Error finding container c23fd44051ca0b4dd82c043678040844f8c67b515adabb9f811f3974952ef1ec: Status 404 returned error can't find the container with id c23fd44051ca0b4dd82c043678040844f8c67b515adabb9f811f3974952ef1ec
Sep 19 09:26:09.451721 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.451681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:09.643665 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.643632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22" event={"ID":"f8730d4c-92c0-4fad-becc-2a066444db6e","Type":"ContainerStarted","Data":"a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486"}
Sep 19 09:26:09.643863 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.643686    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22" event={"ID":"f8730d4c-92c0-4fad-becc-2a066444db6e","Type":"ContainerStarted","Data":"c23fd44051ca0b4dd82c043678040844f8c67b515adabb9f811f3974952ef1ec"}
Sep 19 09:26:09.643962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.643948    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:09.662506 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.662453    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22" podStartSLOduration=1.662435716 podStartE2EDuration="1.662435716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:09.660836208 +0000 UTC m=+1183.347641505" watchObservedRunningTime="2026-09-19 09:26:09.662435716 +0000 UTC m=+1183.349241013"
Sep 19 09:26:09.903025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:09.902919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:10.119584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.119548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:10.217040 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.216952    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:10.397708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.397677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:10.763706 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.763670    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:10.858234 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.858197    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:10.888877 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.888839    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"]
Sep 19 09:26:10.892722 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.892697    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:10.895637 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.895483    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b272f719-dockercfg-x8x5p\""
Sep 19 09:26:10.895637 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.895590    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b272f719\""
Sep 19 09:26:10.909326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.909301    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"]
Sep 19 09:26:10.997056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.997016    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a814630-ba8a-46d8-b873-6132b1d87460-plugins-config-volume\") pod \"payload-processing-e2e-ait-b272f719-6cb988486c-zhj49\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") " pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:10.997056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:10.997059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxfbg\" (UniqueName: \"kubernetes.io/projected/2a814630-ba8a-46d8-b873-6132b1d87460-kube-api-access-xxfbg\") pod \"payload-processing-e2e-ait-b272f719-6cb988486c-zhj49\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") " pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:11.098547 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.098440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a814630-ba8a-46d8-b873-6132b1d87460-plugins-config-volume\") pod \"payload-processing-e2e-ait-b272f719-6cb988486c-zhj49\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") " pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:11.098547 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.098493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxfbg\" (UniqueName: \"kubernetes.io/projected/2a814630-ba8a-46d8-b873-6132b1d87460-kube-api-access-xxfbg\") pod \"payload-processing-e2e-ait-b272f719-6cb988486c-zhj49\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") " pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:11.099237 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.099214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a814630-ba8a-46d8-b873-6132b1d87460-plugins-config-volume\") pod \"payload-processing-e2e-ait-b272f719-6cb988486c-zhj49\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") " pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:11.107004 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.106969    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxfbg\" (UniqueName: \"kubernetes.io/projected/2a814630-ba8a-46d8-b873-6132b1d87460-kube-api-access-xxfbg\") pod \"payload-processing-e2e-ait-b272f719-6cb988486c-zhj49\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") " pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:11.210446 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.210409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:11.301769 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.301737    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:11.484189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.484156    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"]
Sep 19 09:26:11.487582 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:11.487548    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a814630_ba8a_46d8_b873_6132b1d87460.slice/crio-e6619ddbbda66b346c8ff8cad7c37af85d68d0ac19d77681b4c997c42b333d1b WatchSource:0}: Error finding container e6619ddbbda66b346c8ff8cad7c37af85d68d0ac19d77681b4c997c42b333d1b: Status 404 returned error can't find the container with id e6619ddbbda66b346c8ff8cad7c37af85d68d0ac19d77681b4c997c42b333d1b
Sep 19 09:26:11.654605 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.654567    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49" event={"ID":"2a814630-ba8a-46d8-b873-6132b1d87460","Type":"ContainerStarted","Data":"230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1"}
Sep 19 09:26:11.654798 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.654630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49" event={"ID":"2a814630-ba8a-46d8-b873-6132b1d87460","Type":"ContainerStarted","Data":"e6619ddbbda66b346c8ff8cad7c37af85d68d0ac19d77681b4c997c42b333d1b"}
Sep 19 09:26:11.654798 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.654771    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:11.671989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.671912    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49" podStartSLOduration=1.67189027 podStartE2EDuration="1.67189027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:11.671062768 +0000 UTC m=+1185.357868073" watchObservedRunningTime="2026-09-19 09:26:11.67189027 +0000 UTC m=+1185.358695571"
Sep 19 09:26:11.981239 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:11.981204    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:26:12.085527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:12.085486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:12.599530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:12.599496    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:26:13.134385 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:13.134345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:13.237187 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:13.237153    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:13.340796 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:13.340762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:13.434407 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:13.434318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:13.535047 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:13.535008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:14.239565 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:14.239530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:15.220620 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:15.220589    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:15.543627 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:15.543546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:15.624227 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:15.624199    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:26:15.983044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:15.982999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:16.095307 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:16.095266    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:16.198585 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:16.198550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:16.302342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:16.302259    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:16.403533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:16.403500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:16.962261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:16.962228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:17.073564 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:17.073529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:17.182975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:17.182940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:17.568364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:17.568330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:17.667539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:17.667512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:17.863282 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:17.863252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:18.200698 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:18.200647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:18.524173 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:18.524054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:18.623959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:18.623922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:19.021259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:19.021229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:19.122003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:19.121957    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:19.426305 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:19.426268    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:20.468579 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:20.468549    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:20.650977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:20.650949    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:26:20.949997 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:20.949960    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:21.049821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:21.049789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:21.142878 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:21.142840    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:21.394723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:21.394683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:22.250289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:22.250252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:22.346983 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:22.346942    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:22.535145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:22.535032    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:22.660989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:22.660962    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:22.840821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:22.840738    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:22.932742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:22.932694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:23.268881 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.268847    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"]
Sep 19 09:26:23.272513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.272495    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"
Sep 19 09:26:23.275063 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.275040    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:26:23.275210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.275093    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v7qlc\""
Sep 19 09:26:23.281625 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.281592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"]
Sep 19 09:26:23.405030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.404992    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fb6d9d-28bb-4cea-951c-dc1f90c0905e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b272f719-jnxxl\" (UID: \"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"
Sep 19 09:26:23.506255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.506215    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fb6d9d-28bb-4cea-951c-dc1f90c0905e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b272f719-jnxxl\" (UID: \"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"
Sep 19 09:26:23.506895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.506874    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fb6d9d-28bb-4cea-951c-dc1f90c0905e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b272f719-jnxxl\" (UID: \"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"
Sep 19 09:26:23.584504 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.584409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"
Sep 19 09:26:23.598170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.598124    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:23.858173 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:23.858149    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"]
Sep 19 09:26:23.860085 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:23.860058    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4fb6d9d_28bb_4cea_951c_dc1f90c0905e.slice/crio-4afa53f6a75660db9f389d6ba1ea92a78f73d3ee31d3bd95af0f145739d271b9 WatchSource:0}: Error finding container 4afa53f6a75660db9f389d6ba1ea92a78f73d3ee31d3bd95af0f145739d271b9: Status 404 returned error can't find the container with id 4afa53f6a75660db9f389d6ba1ea92a78f73d3ee31d3bd95af0f145739d271b9
Sep 19 09:26:24.365005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:24.364964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:24.465245 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:24.465211    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:24.562356 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:24.562323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:24.707295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:24.707260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:24.719517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:24.719473    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl" event={"ID":"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e","Type":"ContainerStarted","Data":"4afa53f6a75660db9f389d6ba1ea92a78f73d3ee31d3bd95af0f145739d271b9"}
Sep 19 09:26:25.537534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:25.537497    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:26.432965 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:26.432927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:26.730820 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:26.730785    2581 generic.go:356] "Generic (PLEG): container finished" podID="b4fb6d9d-28bb-4cea-951c-dc1f90c0905e" containerID="dcff64f6c63f8736952168dd3bd8a571dbf0120cfc7cf078ec8cb77647f025a6" exitCode=0
Sep 19 09:26:26.731174 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:26.730868    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl" event={"ID":"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e","Type":"ContainerDied","Data":"dcff64f6c63f8736952168dd3bd8a571dbf0120cfc7cf078ec8cb77647f025a6"}
Sep 19 09:26:27.345997 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:27.345961    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:27.916590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:27.916560    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"
Sep 19 09:26:28.049759 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:28.049662    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fb6d9d-28bb-4cea-951c-dc1f90c0905e-maas-api-service-ca\") pod \"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e\" (UID: \"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e\") "
Sep 19 09:26:28.050003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:28.049978    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4fb6d9d-28bb-4cea-951c-dc1f90c0905e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4fb6d9d-28bb-4cea-951c-dc1f90c0905e" (UID: "b4fb6d9d-28bb-4cea-951c-dc1f90c0905e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:26:28.150566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:28.150528    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4fb6d9d-28bb-4cea-951c-dc1f90c0905e-maas-api-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:28.455778 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:28.455739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:28.741696 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:28.741615    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"
Sep 19 09:26:28.741696 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:28.741654    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl" event={"ID":"b4fb6d9d-28bb-4cea-951c-dc1f90c0905e","Type":"ContainerDied","Data":"4afa53f6a75660db9f389d6ba1ea92a78f73d3ee31d3bd95af0f145739d271b9"}
Sep 19 09:26:28.741897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:28.741739    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4afa53f6a75660db9f389d6ba1ea92a78f73d3ee31d3bd95af0f145739d271b9"
Sep 19 09:26:29.253581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:29.253547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:30.117837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:30.117795    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:30.791223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:30.791184    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:31.445925 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:31.445884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:31.546309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:31.546274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:33.996708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:33.996665    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:34.023183 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.023148    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"]
Sep 19 09:26:34.023482 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.023454    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49" podUID="2a814630-ba8a-46d8-b873-6132b1d87460" containerName="payload-processing" containerID="cri-o://230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1" gracePeriod=30
Sep 19 09:26:34.095434 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.095394    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:34.198916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.198872    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:34.201256 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.201184    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"]
Sep 19 09:26:34.204500 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.204471    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b272f719-jnxxl"]
Sep 19 09:26:34.294410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.294368    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:34.296135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.296085    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:34.336215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.336181    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"]
Sep 19 09:26:34.336472 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.336439    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z" podUID="e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" containerName="istio-proxy" containerID="cri-o://63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9" gracePeriod=30
Sep 19 09:26:34.368898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.368854    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:34.406791 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.406762    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a814630-ba8a-46d8-b873-6132b1d87460-plugins-config-volume\") pod \"2a814630-ba8a-46d8-b873-6132b1d87460\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") "
Sep 19 09:26:34.406935 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.406831    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxfbg\" (UniqueName: \"kubernetes.io/projected/2a814630-ba8a-46d8-b873-6132b1d87460-kube-api-access-xxfbg\") pod \"2a814630-ba8a-46d8-b873-6132b1d87460\" (UID: \"2a814630-ba8a-46d8-b873-6132b1d87460\") "
Sep 19 09:26:34.407282 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.407254    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a814630-ba8a-46d8-b873-6132b1d87460-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2a814630-ba8a-46d8-b873-6132b1d87460" (UID: "2a814630-ba8a-46d8-b873-6132b1d87460"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:26:34.409379 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.409351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a814630-ba8a-46d8-b873-6132b1d87460-kube-api-access-xxfbg" (OuterVolumeSpecName: "kube-api-access-xxfbg") pod "2a814630-ba8a-46d8-b873-6132b1d87460" (UID: "2a814630-ba8a-46d8-b873-6132b1d87460"). InnerVolumeSpecName "kube-api-access-xxfbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:26:34.507393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.507355    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxfbg\" (UniqueName: \"kubernetes.io/projected/2a814630-ba8a-46d8-b873-6132b1d87460-kube-api-access-xxfbg\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:34.507393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.507386    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a814630-ba8a-46d8-b873-6132b1d87460-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:34.698060 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.698021    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:34.773643 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.773604    2581 generic.go:356] "Generic (PLEG): container finished" podID="2a814630-ba8a-46d8-b873-6132b1d87460" containerID="230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1" exitCode=0
Sep 19 09:26:34.773827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.773651    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"
Sep 19 09:26:34.773827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.773693    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49" event={"ID":"2a814630-ba8a-46d8-b873-6132b1d87460","Type":"ContainerDied","Data":"230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1"}
Sep 19 09:26:34.773827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.773750    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49" event={"ID":"2a814630-ba8a-46d8-b873-6132b1d87460","Type":"ContainerDied","Data":"e6619ddbbda66b346c8ff8cad7c37af85d68d0ac19d77681b4c997c42b333d1b"}
Sep 19 09:26:34.773827 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.773779    2581 scope.go:117] "RemoveContainer" containerID="230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1"
Sep 19 09:26:34.784339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.784320    2581 scope.go:117] "RemoveContainer" containerID="230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1"
Sep 19 09:26:34.784631 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:34.784609    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1\": container with ID starting with 230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1 not found: ID does not exist" containerID="230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1"
Sep 19 09:26:34.784695 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.784644    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1"} err="failed to get container status \"230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1\": rpc error: code = NotFound desc = could not find container \"230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1\": container with ID starting with 230c3a4772234420deb972bd729a3a43006a00b7a81a3e53b1f5e21a83ff6be1 not found: ID does not exist"
Sep 19 09:26:34.798523 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.798491    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"]
Sep 19 09:26:34.811341 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.811307    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b272f719-6cb988486c-zhj49"]
Sep 19 09:26:34.832271 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.832241    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a814630-ba8a-46d8-b873-6132b1d87460" path="/var/lib/kubelet/pods/2a814630-ba8a-46d8-b873-6132b1d87460/volumes"
Sep 19 09:26:34.832576 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.832564    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4fb6d9d-28bb-4cea-951c-dc1f90c0905e" path="/var/lib/kubelet/pods/b4fb6d9d-28bb-4cea-951c-dc1f90c0905e/volumes"
Sep 19 09:26:34.951084 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:34.950983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:35.084163 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.084122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:35.186317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.186269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:35.366191 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.366116    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"]
Sep 19 09:26:35.366474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.366461    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4fb6d9d-28bb-4cea-951c-dc1f90c0905e" containerName="revoke-keys"
Sep 19 09:26:35.366530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.366476    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4fb6d9d-28bb-4cea-951c-dc1f90c0905e" containerName="revoke-keys"
Sep 19 09:26:35.366530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.366499    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a814630-ba8a-46d8-b873-6132b1d87460" containerName="payload-processing"
Sep 19 09:26:35.366530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.366504    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a814630-ba8a-46d8-b873-6132b1d87460" containerName="payload-processing"
Sep 19 09:26:35.366640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.366561    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4fb6d9d-28bb-4cea-951c-dc1f90c0905e" containerName="revoke-keys"
Sep 19 09:26:35.366640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.366570    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a814630-ba8a-46d8-b873-6132b1d87460" containerName="payload-processing"
Sep 19 09:26:35.371230 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.371207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.374030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.373998    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d6a0ebf6-gw-openshift-default-dockercfg-mzhpj\""
Sep 19 09:26:35.388217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.388187    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"]
Sep 19 09:26:35.414363 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.414330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:35.516491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516448    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-envoy\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.516697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lnmt\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-kube-api-access-5lnmt\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.516697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-data\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.516697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-credential-socket\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.516697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-certs\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.516697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516648    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-token\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.516697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516682    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-socket\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.517058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-podinfo\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.517058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.516793    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istiod-ca-cert\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617265 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lnmt\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-kube-api-access-5lnmt\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617265 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-data\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617265 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-credential-socket\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617265 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617252    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-certs\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617265 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-token\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-socket\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617532    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-podinfo\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istiod-ca-cert\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617643    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-certs\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.617751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617680    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-envoy\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.618018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617917    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-credential-socket\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.618018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.617968    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-socket\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.618018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.618004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-data\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.618357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.618334    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istiod-ca-cert\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.620072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.620049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-podinfo\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.620211 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.620081    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-envoy\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.628933 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.628903    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-token\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.629168 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.629149    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lnmt\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-kube-api-access-5lnmt\") pod \"e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") " pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.681281 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.681238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:35.699126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.698982    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:35.783684 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.783651    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:26:35.785921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.785895    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:35.971350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.971324    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"]
Sep 19 09:26:35.973471 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:35.973443    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf50f4adf_6d6b_4498_9e5a_45c6c8d2b769.slice/crio-4f1872261438a293b53402e171e32ef2403e5ba4c37485454d63c8ced0480c73 WatchSource:0}: Error finding container 4f1872261438a293b53402e171e32ef2403e5ba4c37485454d63c8ced0480c73: Status 404 returned error can't find the container with id 4f1872261438a293b53402e171e32ef2403e5ba4c37485454d63c8ced0480c73
Sep 19 09:26:35.975694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.975644    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:26:35.975794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.975757    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:26:35.975859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:35.975803    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:26:36.678480 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:36.678444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:36.788938 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:36.788904    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" event={"ID":"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769","Type":"ContainerStarted","Data":"2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2"}
Sep 19 09:26:36.789087 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:36.788998    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" event={"ID":"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769","Type":"ContainerStarted","Data":"4f1872261438a293b53402e171e32ef2403e5ba4c37485454d63c8ced0480c73"}
Sep 19 09:26:36.813244 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:36.813189    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podStartSLOduration=1.813173138 podStartE2EDuration="1.813173138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:36.812314123 +0000 UTC m=+1210.499119421" watchObservedRunningTime="2026-09-19 09:26:36.813173138 +0000 UTC m=+1210.499978437"
Sep 19 09:26:36.991868 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:36.991843    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:37.131672 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-envoy\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.131844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131687    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-data\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.131844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131724    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-credential-socket\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.131844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131766    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istiod-ca-cert\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.131844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131806    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-podinfo\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.131844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131842    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-token\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.132274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131881    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-socket\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.132274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.131909    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-certs\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.132274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132009    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:26:37.132274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132078    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmpp6\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-kube-api-access-dmpp6\") pod \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\" (UID: \"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b\") "
Sep 19 09:26:37.132274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132120    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:26:37.132274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132231    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:26:37.132584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132272    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:26:37.132667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132558    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istiod-ca-cert\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.132898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132672    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.132898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132688    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-data\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.132898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132702    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-credential-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.132898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.132860    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:26:37.134254 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.134228    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:26:37.134349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.134270    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:26:37.134349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.134313    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 09:26:37.134533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.134517    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-kube-api-access-dmpp6" (OuterVolumeSpecName: "kube-api-access-dmpp6") pod "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" (UID: "e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b"). InnerVolumeSpecName "kube-api-access-dmpp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:26:37.233726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.233641    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-podinfo\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.233726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.233669    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.233726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.233677    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-workload-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.233726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.233687    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmpp6\" (UniqueName: \"kubernetes.io/projected/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-kube-api-access-dmpp6\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.233726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.233696    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b-istio-envoy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:26:37.681696 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.681662    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:37.686775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.686750    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:37.759756 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.759723    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:37.794838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.794803    2581 generic.go:356] "Generic (PLEG): container finished" podID="e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" containerID="63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9" exitCode=0
Sep 19 09:26:37.794838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.794841    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"
Sep 19 09:26:37.795034 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.794875    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z" event={"ID":"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b","Type":"ContainerDied","Data":"63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9"}
Sep 19 09:26:37.795034 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.794919    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z" event={"ID":"e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b","Type":"ContainerDied","Data":"f2f8229d6d21dd004d677bebbffebda87d2af0f1373fa3b056d1b4bc68120f70"}
Sep 19 09:26:37.795034 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.794943    2581 scope.go:117] "RemoveContainer" containerID="63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9"
Sep 19 09:26:37.795478 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.795452    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:37.796706 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.796579    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:26:37.804666 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.804648    2581 scope.go:117] "RemoveContainer" containerID="63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9"
Sep 19 09:26:37.804987 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:37.804962    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9\": container with ID starting with 63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9 not found: ID does not exist" containerID="63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9"
Sep 19 09:26:37.805082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.805027    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9"} err="failed to get container status \"63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9\": rpc error: code = NotFound desc = could not find container \"63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9\": container with ID starting with 63348d5121882cba094b546e2a9767256c988265a87e11dac4aff4475b15abf9 not found: ID does not exist"
Sep 19 09:26:37.846068 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.846028    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"]
Sep 19 09:26:37.852131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:37.852083    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b272f719-openshift-default-cc499f476-kpm2z"]
Sep 19 09:26:38.343852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:38.343818    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:38.444581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:38.444538    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:38.830965 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:38.830933    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" path="/var/lib/kubelet/pods/e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b/volumes"
Sep 19 09:26:38.938053 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:38.938013    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:39.908352 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:39.908322    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:40.660348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:40.660307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:40.768617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:40.768551    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:41.512044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:41.512009    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:42.292648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.292607    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"]
Sep 19 09:26:42.292967 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.292954    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" containerName="istio-proxy"
Sep 19 09:26:42.293017 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.292969    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" containerName="istio-proxy"
Sep 19 09:26:42.293058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.293035    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0cc9ea9-a2f6-45c0-bc46-67b89752ea9b" containerName="istio-proxy"
Sep 19 09:26:42.297397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.297374    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.300794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.300771    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:26:42.300970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.300806    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d6a0ebf6-serving-cert\""
Sep 19 09:26:42.300970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.300850    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 09:26:42.300970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.300853    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 09:26:42.300970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.300805    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzltl\""
Sep 19 09:26:42.309755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.309732    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"]
Sep 19 09:26:42.327731 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.327694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:42.380940 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.380906    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-metrics-certs\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.381145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.380950    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.381145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.380977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4gn2\" (UniqueName: \"kubernetes.io/projected/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-kube-api-access-w4gn2\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.381145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.381047    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.430307 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.430253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:42.482369 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.482338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-metrics-certs\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.482541 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.482382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.482541 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.482426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4gn2\" (UniqueName: \"kubernetes.io/projected/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-kube-api-access-w4gn2\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.482541 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.482448    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.482652 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:42.482587    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-serving-cert: secret "maas-api-e2e-migrate-d6a0ebf6-serving-cert" not found
Sep 19 09:26:42.482690 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:26:42.482671    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls podName:86f4375a-ca06-4c3c-ba5c-9a94488fa3dc nodeName:}" failed. No retries permitted until 2026-09-19 09:26:42.982650169 +0000 UTC m=+1216.669455445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls") pod "maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb" (UID: "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc") : secret "maas-api-e2e-migrate-d6a0ebf6-serving-cert" not found
Sep 19 09:26:42.483278 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.483176    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-openshift-service-ca\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.484964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.484940    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-metrics-certs\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.490487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.490460    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4gn2\" (UniqueName: \"kubernetes.io/projected/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-kube-api-access-w4gn2\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.830310 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.830269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:42.938562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.938529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:42.949187 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.949156    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"]
Sep 19 09:26:42.952832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.952814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:42.955683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.955660    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-d6a0ebf6-dockercfg-nfd6c\""
Sep 19 09:26:42.955683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.955684    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-d6a0ebf6\""
Sep 19 09:26:42.963816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.963792    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"]
Sep 19 09:26:42.980836 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.980800    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"]
Sep 19 09:26:42.984088 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.984062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:42.988018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.987998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:42.990704 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:42.990681    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls\") pod \"maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:43.000695 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.000670    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"]
Sep 19 09:26:43.034997 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.034959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:43.089221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.089128    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqrb9\" (UniqueName: \"kubernetes.io/projected/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-kube-api-access-kqrb9\") pod \"payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.089221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.089175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49f6759a-f9af-4c27-bef3-e523dc5b50b6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.089444 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.089241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6md4\" (UniqueName: \"kubernetes.io/projected/49f6759a-f9af-4c27-bef3-e523dc5b50b6-kube-api-access-t6md4\") pod \"payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.089444 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.089281    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.190332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.190279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49f6759a-f9af-4c27-bef3-e523dc5b50b6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.190332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.190338    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6md4\" (UniqueName: \"kubernetes.io/projected/49f6759a-f9af-4c27-bef3-e523dc5b50b6-kube-api-access-t6md4\") pod \"payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.190563 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.190369    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.190563 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.190504    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqrb9\" (UniqueName: \"kubernetes.io/projected/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-kube-api-access-kqrb9\") pod \"payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.191148 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.191093    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49f6759a-f9af-4c27-bef3-e523dc5b50b6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.191192 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.191150    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.199450 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.199424    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqrb9\" (UniqueName: \"kubernetes.io/projected/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-kube-api-access-kqrb9\") pod \"payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.199594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.199455    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6md4\" (UniqueName: \"kubernetes.io/projected/49f6759a-f9af-4c27-bef3-e523dc5b50b6-kube-api-access-t6md4\") pod \"payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") " pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.208519 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.208492    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:43.253228 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.253194    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"]
Sep 19 09:26:43.257690 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.257657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.260292 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.260221    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"default-dockercfg-sv6sv\""
Sep 19 09:26:43.260557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.260427    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"simulator-w0-14e3f6-kserve-self-signed-certs\""
Sep 19 09:26:43.260557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.260548    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"openshift-service-ca.crt\""
Sep 19 09:26:43.260675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.260640    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"kube-root-ca.crt\""
Sep 19 09:26:43.264015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.263994    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.272290 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.272259    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"]
Sep 19 09:26:43.302136 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.302064    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.302799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.302768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:43.392686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.392657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrsrg\" (UniqueName: \"kubernetes.io/projected/36aa4868-42ef-4a86-ba60-bd1d18c19d06-kube-api-access-mrsrg\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.392838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.392715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tls-certs\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.392838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.392759    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-dshm\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.392838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.392778    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-model-cache\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.392838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.392817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tmp-dir\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.392995 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.392903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-home\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.484781 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.484726    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"]
Sep 19 09:26:43.487218 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:43.487189    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f4375a_ca06_4c3c_ba5c_9a94488fa3dc.slice/crio-e3de8e9729a04756dd19f4386dd5086cd70512ef467cba7e50b5d5f52d78ffac WatchSource:0}: Error finding container e3de8e9729a04756dd19f4386dd5086cd70512ef467cba7e50b5d5f52d78ffac: Status 404 returned error can't find the container with id e3de8e9729a04756dd19f4386dd5086cd70512ef467cba7e50b5d5f52d78ffac
Sep 19 09:26:43.493463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tls-certs\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.493552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-dshm\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.493552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-model-cache\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.493661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493561    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tmp-dir\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.493661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493600    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-home\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.493661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrsrg\" (UniqueName: \"kubernetes.io/projected/36aa4868-42ef-4a86-ba60-bd1d18c19d06-kube-api-access-mrsrg\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.493993 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493965    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-model-cache\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.493993 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.493989    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-home\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.494185 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.494040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tmp-dir\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.495965 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.495939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-dshm\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.496231 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.496214    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tls-certs\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.501273 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.501252    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrsrg\" (UniqueName: \"kubernetes.io/projected/36aa4868-42ef-4a86-ba60-bd1d18c19d06-kube-api-access-mrsrg\") pod \"simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") " pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.549278 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.549252    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"]
Sep 19 09:26:43.551316 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:43.551287    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8d5218_cf14_4785_8e2f_55e2c1bb68ae.slice/crio-45b1eacad1b030511a9e49facc3645878ee9f7222309d115c086cf704d538ed9 WatchSource:0}: Error finding container 45b1eacad1b030511a9e49facc3645878ee9f7222309d115c086cf704d538ed9: Status 404 returned error can't find the container with id 45b1eacad1b030511a9e49facc3645878ee9f7222309d115c086cf704d538ed9
Sep 19 09:26:43.571579 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.571545    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:43.572533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.572504    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:43.583052 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.583031    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"]
Sep 19 09:26:43.829269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.829231    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn" event={"ID":"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae","Type":"ContainerStarted","Data":"6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88"}
Sep 19 09:26:43.829471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.829330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn" event={"ID":"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae","Type":"ContainerStarted","Data":"45b1eacad1b030511a9e49facc3645878ee9f7222309d115c086cf704d538ed9"}
Sep 19 09:26:43.829471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.829359    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:43.831044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.831005    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj" event={"ID":"49f6759a-f9af-4c27-bef3-e523dc5b50b6","Type":"ContainerStarted","Data":"b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1"}
Sep 19 09:26:43.831486 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.831059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj" event={"ID":"49f6759a-f9af-4c27-bef3-e523dc5b50b6","Type":"ContainerStarted","Data":"9a8dca5d5c7e6ff7e298d4850dda5471a6d42fc72de3483dfa0080d9b0c5bbb8"}
Sep 19 09:26:43.831486 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.831120    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:26:43.832421 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.832397    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb" event={"ID":"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc","Type":"ContainerStarted","Data":"e3de8e9729a04756dd19f4386dd5086cd70512ef467cba7e50b5d5f52d78ffac"}
Sep 19 09:26:43.849261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.849205    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn" podStartSLOduration=1.849188697 podStartE2EDuration="1.849188697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:43.84788421 +0000 UTC m=+1217.534689523" watchObservedRunningTime="2026-09-19 09:26:43.849188697 +0000 UTC m=+1217.535994015"
Sep 19 09:26:43.856179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.856157    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"]
Sep 19 09:26:43.858804 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:26:43.858771    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36aa4868_42ef_4a86_ba60_bd1d18c19d06.slice/crio-51d7c14bba22210e55f5398a43cff935d0b46c0589b91028224c278aa05442ea WatchSource:0}: Error finding container 51d7c14bba22210e55f5398a43cff935d0b46c0589b91028224c278aa05442ea: Status 404 returned error can't find the container with id 51d7c14bba22210e55f5398a43cff935d0b46c0589b91028224c278aa05442ea
Sep 19 09:26:43.867266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:43.867202    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj" podStartSLOduration=1.867183222 podStartE2EDuration="1.867183222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:43.866728142 +0000 UTC m=+1217.553533435" watchObservedRunningTime="2026-09-19 09:26:43.867183222 +0000 UTC m=+1217.553988527"
Sep 19 09:26:44.001326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:44.001289    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:44.808755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:44.808720    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:26:44.840181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:44.840137    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm" event={"ID":"36aa4868-42ef-4a86-ba60-bd1d18c19d06","Type":"ContainerStarted","Data":"442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a"}
Sep 19 09:26:44.840181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:44.840196    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm" event={"ID":"36aa4868-42ef-4a86-ba60-bd1d18c19d06","Type":"ContainerStarted","Data":"51d7c14bba22210e55f5398a43cff935d0b46c0589b91028224c278aa05442ea"}
Sep 19 09:26:44.860492 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:44.860440    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm" podStartSLOduration=1.860420907 podStartE2EDuration="1.860420907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:44.858070283 +0000 UTC m=+1218.544875584" watchObservedRunningTime="2026-09-19 09:26:44.860420907 +0000 UTC m=+1218.547226206"
Sep 19 09:26:45.848777 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:45.848745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb" event={"ID":"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc","Type":"ContainerStarted","Data":"2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344"}
Sep 19 09:26:45.849198 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:45.848858    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:45.868845 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:45.868793    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb" podStartSLOduration=1.6057859030000001 podStartE2EDuration="3.868776299s" totalImagesPullingTime="2.262990396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:26:45.866604237 +0000 UTC m=+1219.553409546" watchObservedRunningTime="2026-09-19 09:26:45.868776299 +0000 UTC m=+1219.555581597"
Sep 19 09:26:51.090197 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:51.090160    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:26:51.858991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:51.858959    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:26:53.572458 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:53.572419    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:53.572929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:53.572504    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:53.602687 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:53.602654    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:53.919060 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:53.919030    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:26:54.842663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:54.842632    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:26:54.843167 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:26:54.842712    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:27:02.948277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:02.948240    2581 generic.go:356] "Generic (PLEG): container finished" podID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerID="2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2" exitCode=0
Sep 19 09:27:02.948708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:02.948296    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" event={"ID":"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769","Type":"ContainerDied","Data":"2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2"}
Sep 19 09:27:02.948708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:02.948664    2581 scope.go:117] "RemoveContainer" containerID="2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2"
Sep 19 09:27:02.949809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:02.949778    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:02.949923 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:02.949868    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:02.949923 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:02.949914    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:03.237856 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:03.237771    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:03.955967 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:03.955936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" event={"ID":"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769","Type":"ContainerStarted","Data":"c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574"}
Sep 19 09:27:04.604852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:04.604815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:04.682485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:04.682447    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:27:04.683812 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:04.683781    2581 patch_prober.go:28] interesting pod/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 19 09:27:04.684073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:04.684042    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 19 09:27:05.426531 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:05.426476    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:05.682381 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:05.682280    2581 patch_prober.go:28] interesting pod/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 19 09:27:05.682381 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:05.682344    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 19 09:27:06.222311 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:06.222267    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:06.681946 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:06.681906    2581 patch_prober.go:28] interesting pod/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 19 09:27:06.682366 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:06.681970    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 19 09:27:06.997259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:06.997150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:07.682339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:07.682302    2581 patch_prober.go:28] interesting pod/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 19 09:27:07.682754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:07.682372    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 19 09:27:07.764153 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:07.764078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:07.795756 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:07.795702    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:27:08.682662 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:08.682625    2581 patch_prober.go:28] interesting pod/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Sep 19 09:27:08.683060 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:08.682690    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Sep 19 09:27:08.710189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:08.710141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:09.605332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:09.605288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:09.686636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:09.686601    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:27:09.687586 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:09.687566    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:27:10.293758 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:10.293703    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:11.088654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:11.088618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:11.765720 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:11.765678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:12.538349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:12.538310    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:13.312671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.312628    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:13.365889 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.365853    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"]
Sep 19 09:27:13.369639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.369623    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.372830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.372806    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-acc640\"/\"openshift-service-ca.crt\""
Sep 19 09:27:13.372972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.372832    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-acc640\"/\"default-dockercfg-xrxn4\""
Sep 19 09:27:13.372972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.372841    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-acc640\"/\"kube-root-ca.crt\""
Sep 19 09:27:13.372972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.372878    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-acc640\"/\"simulator-w4-acc640-kserve-self-signed-certs\""
Sep 19 09:27:13.380259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.380234    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"]
Sep 19 09:27:13.471090 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.471058    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:13.496818 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.496785    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-home\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.496973 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.496830    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29mww\" (UniqueName: \"kubernetes.io/projected/c7696837-5859-4494-9d90-2e0099b03c7d-kube-api-access-29mww\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.496973 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.496898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-model-cache\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.496973 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.496952    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7696837-5859-4494-9d90-2e0099b03c7d-tls-certs\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.497080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.496986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-tmp-dir\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.497080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.497015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-dshm\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.597655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.597515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-model-cache\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.597655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.597571    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7696837-5859-4494-9d90-2e0099b03c7d-tls-certs\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.597655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.597640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-tmp-dir\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.598277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.597670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-dshm\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.598277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.597723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-home\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.598277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.597777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29mww\" (UniqueName: \"kubernetes.io/projected/c7696837-5859-4494-9d90-2e0099b03c7d-kube-api-access-29mww\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.598277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.597974    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-model-cache\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.598277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.598050    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-tmp-dir\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.598277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.598174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-home\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.600020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.600002    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-dshm\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.600296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.600281    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7696837-5859-4494-9d90-2e0099b03c7d-tls-certs\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.606921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.606899    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29mww\" (UniqueName: \"kubernetes.io/projected/c7696837-5859-4494-9d90-2e0099b03c7d-kube-api-access-29mww\") pod \"simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") " pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.679726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.679681    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:13.957873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:13.957837    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"]
Sep 19 09:27:13.961444 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:27:13.961413    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7696837_5859_4494_9d90_2e0099b03c7d.slice/crio-de4675a49cf316bd693aef4ee1a311452eab144d7d67a87d733dfc1228965cd8 WatchSource:0}: Error finding container de4675a49cf316bd693aef4ee1a311452eab144d7d67a87d733dfc1228965cd8: Status 404 returned error can't find the container with id de4675a49cf316bd693aef4ee1a311452eab144d7d67a87d733dfc1228965cd8
Sep 19 09:27:14.008437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:14.008396    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8" event={"ID":"c7696837-5859-4494-9d90-2e0099b03c7d","Type":"ContainerStarted","Data":"de4675a49cf316bd693aef4ee1a311452eab144d7d67a87d733dfc1228965cd8"}
Sep 19 09:27:14.055758 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:14.055708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:15.015658 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:15.015627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8" event={"ID":"c7696837-5859-4494-9d90-2e0099b03c7d","Type":"ContainerStarted","Data":"ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e"}
Sep 19 09:27:15.033134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:15.033014    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8" podStartSLOduration=2.032994074 podStartE2EDuration="2.032994074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:15.032525442 +0000 UTC m=+1248.719330741" watchObservedRunningTime="2026-09-19 09:27:15.032994074 +0000 UTC m=+1248.719799374"
Sep 19 09:27:19.078694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:19.078658    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:27:19.619160 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:19.619123    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:20.430309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.430276    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"]
Sep 19 09:27:20.430815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.430606    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb" podUID="86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" containerName="maas-api" containerID="cri-o://2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344" gracePeriod=30
Sep 19 09:27:20.466976 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.466938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:20.492940 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.492896    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"]
Sep 19 09:27:20.493232 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.493207    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj" podUID="49f6759a-f9af-4c27-bef3-e523dc5b50b6" containerName="payload-processing" containerID="cri-o://b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1" gracePeriod=30
Sep 19 09:27:20.498247 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.498197    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"]
Sep 19 09:27:20.498465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.498426    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn" podUID="bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" containerName="payload-pre-processing" containerID="cri-o://6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88" gracePeriod=30
Sep 19 09:27:20.555070 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.555036    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:20.655496 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.655455    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:20.748485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.748452    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:27:20.752985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.752960    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:27:20.757159 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.757089    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:20.788524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.788494    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:27:20.820512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.820482    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"]
Sep 19 09:27:20.820706 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.820682    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy" containerID="cri-o://c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574" gracePeriod=30
Sep 19 09:27:20.849338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.849305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:20.863302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863278    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6md4\" (UniqueName: \"kubernetes.io/projected/49f6759a-f9af-4c27-bef3-e523dc5b50b6-kube-api-access-t6md4\") pod \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") "
Sep 19 09:27:20.863441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-metrics-certs\") pod \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") "
Sep 19 09:27:20.863441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863348    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqrb9\" (UniqueName: \"kubernetes.io/projected/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-kube-api-access-kqrb9\") pod \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") "
Sep 19 09:27:20.863441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863373    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4gn2\" (UniqueName: \"kubernetes.io/projected/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-kube-api-access-w4gn2\") pod \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") "
Sep 19 09:27:20.863441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863388    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49f6759a-f9af-4c27-bef3-e523dc5b50b6-plugins-config-volume\") pod \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\" (UID: \"49f6759a-f9af-4c27-bef3-e523dc5b50b6\") "
Sep 19 09:27:20.863441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863408    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-openshift-service-ca\") pod \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") "
Sep 19 09:27:20.863441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863433    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls\") pod \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\" (UID: \"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc\") "
Sep 19 09:27:20.863812 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863544    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-plugins-config-volume\") pod \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\" (UID: \"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae\") "
Sep 19 09:27:20.863871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863837    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49f6759a-f9af-4c27-bef3-e523dc5b50b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "49f6759a-f9af-4c27-bef3-e523dc5b50b6" (UID: "49f6759a-f9af-4c27-bef3-e523dc5b50b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:27:20.863926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.863841    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" (UID: "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:27:20.864394 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.864363    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" (UID: "bb8d5218-cf14-4785-8e2f-55e2c1bb68ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:27:20.865970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.865944    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-kube-api-access-w4gn2" (OuterVolumeSpecName: "kube-api-access-w4gn2") pod "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" (UID: "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc"). InnerVolumeSpecName "kube-api-access-w4gn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:20.866072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.865970    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" (UID: "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:27:20.866072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.866045    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-kube-api-access-kqrb9" (OuterVolumeSpecName: "kube-api-access-kqrb9") pod "bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" (UID: "bb8d5218-cf14-4785-8e2f-55e2c1bb68ae"). InnerVolumeSpecName "kube-api-access-kqrb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:20.866220 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.866201    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49f6759a-f9af-4c27-bef3-e523dc5b50b6-kube-api-access-t6md4" (OuterVolumeSpecName: "kube-api-access-t6md4") pod "49f6759a-f9af-4c27-bef3-e523dc5b50b6" (UID: "49f6759a-f9af-4c27-bef3-e523dc5b50b6"). InnerVolumeSpecName "kube-api-access-t6md4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:20.866318 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.866299    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" (UID: "86f4375a-ca06-4c3c-ba5c-9a94488fa3dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964588    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964641    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964655    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964666    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6md4\" (UniqueName: \"kubernetes.io/projected/49f6759a-f9af-4c27-bef3-e523dc5b50b6-kube-api-access-t6md4\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964678    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964689    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqrb9\" (UniqueName: \"kubernetes.io/projected/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae-kube-api-access-kqrb9\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964699    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4gn2\" (UniqueName: \"kubernetes.io/projected/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc-kube-api-access-w4gn2\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:20.964711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:20.964711    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/49f6759a-f9af-4c27-bef3-e523dc5b50b6-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:21.047588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.047553    2581 generic.go:356] "Generic (PLEG): container finished" podID="bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" containerID="6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88" exitCode=0
Sep 19 09:27:21.047784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.047597    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"
Sep 19 09:27:21.047784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.047642    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn" event={"ID":"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae","Type":"ContainerDied","Data":"6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88"}
Sep 19 09:27:21.047784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.047707    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn" event={"ID":"bb8d5218-cf14-4785-8e2f-55e2c1bb68ae","Type":"ContainerDied","Data":"45b1eacad1b030511a9e49facc3645878ee9f7222309d115c086cf704d538ed9"}
Sep 19 09:27:21.047784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.047737    2581 scope.go:117] "RemoveContainer" containerID="6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88"
Sep 19 09:27:21.049159 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.049093    2581 generic.go:356] "Generic (PLEG): container finished" podID="49f6759a-f9af-4c27-bef3-e523dc5b50b6" containerID="b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1" exitCode=0
Sep 19 09:27:21.049277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.049148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj" event={"ID":"49f6759a-f9af-4c27-bef3-e523dc5b50b6","Type":"ContainerDied","Data":"b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1"}
Sep 19 09:27:21.049277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.049214    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj" event={"ID":"49f6759a-f9af-4c27-bef3-e523dc5b50b6","Type":"ContainerDied","Data":"9a8dca5d5c7e6ff7e298d4850dda5471a6d42fc72de3483dfa0080d9b0c5bbb8"}
Sep 19 09:27:21.049277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.049159    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"
Sep 19 09:27:21.050932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.050784    2581 generic.go:356] "Generic (PLEG): container finished" podID="86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" containerID="2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344" exitCode=0
Sep 19 09:27:21.050932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.050834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb" event={"ID":"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc","Type":"ContainerDied","Data":"2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344"}
Sep 19 09:27:21.050932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.050887    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb" event={"ID":"86f4375a-ca06-4c3c-ba5c-9a94488fa3dc","Type":"ContainerDied","Data":"e3de8e9729a04756dd19f4386dd5086cd70512ef467cba7e50b5d5f52d78ffac"}
Sep 19 09:27:21.050932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.050838    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"
Sep 19 09:27:21.058982 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.058961    2581 scope.go:117] "RemoveContainer" containerID="6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88"
Sep 19 09:27:21.059319 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:21.059293    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88\": container with ID starting with 6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88 not found: ID does not exist" containerID="6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88"
Sep 19 09:27:21.059423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.059330    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88"} err="failed to get container status \"6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88\": rpc error: code = NotFound desc = could not find container \"6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88\": container with ID starting with 6ef8ef5aa5c2029d495f974bb3ad61c8782e6c18c36351515204232b2e665d88 not found: ID does not exist"
Sep 19 09:27:21.059423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.059348    2581 scope.go:117] "RemoveContainer" containerID="b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1"
Sep 19 09:27:21.068259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.068241    2581 scope.go:117] "RemoveContainer" containerID="b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1"
Sep 19 09:27:21.068552 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:21.068535    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1\": container with ID starting with b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1 not found: ID does not exist" containerID="b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1"
Sep 19 09:27:21.068623 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.068559    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1"} err="failed to get container status \"b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1\": rpc error: code = NotFound desc = could not find container \"b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1\": container with ID starting with b9f9cb9e6813a42cf9e5d94ebad8b09258d981ac23dbe8eed4b7756992af4cc1 not found: ID does not exist"
Sep 19 09:27:21.068623 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.068575    2581 scope.go:117] "RemoveContainer" containerID="2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344"
Sep 19 09:27:21.077046 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.077018    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"]
Sep 19 09:27:21.077742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.077727    2581 scope.go:117] "RemoveContainer" containerID="2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344"
Sep 19 09:27:21.078070 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:21.078052    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344\": container with ID starting with 2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344 not found: ID does not exist" containerID="2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344"
Sep 19 09:27:21.078157 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.078079    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344"} err="failed to get container status \"2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344\": rpc error: code = NotFound desc = could not find container \"2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344\": container with ID starting with 2085863475c727577ae5ccb6640162bdb05f8debed6342f59ddda376b5e7d344 not found: ID does not exist"
Sep 19 09:27:21.080723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.080701    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-d6a0ebf6-cfc787c9d-cx7kn"]
Sep 19 09:27:21.090698 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.090675    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"]
Sep 19 09:27:21.097134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.097084    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d6a0ebf6-7bf488686-99rjj"]
Sep 19 09:27:21.112077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.112047    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"]
Sep 19 09:27:21.116481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.116458    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d6a0ebf6-9bdf8549c-7d4gb"]
Sep 19 09:27:21.181286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.181236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:21.303168 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.303041    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:21.610766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.610689    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"]
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611008    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" containerName="maas-api"
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611019    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" containerName="maas-api"
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611030    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49f6759a-f9af-4c27-bef3-e523dc5b50b6" containerName="payload-processing"
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611035    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="49f6759a-f9af-4c27-bef3-e523dc5b50b6" containerName="payload-processing"
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611047    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" containerName="payload-pre-processing"
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611052    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" containerName="payload-pre-processing"
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611126    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" containerName="payload-pre-processing"
Sep 19 09:27:21.611131 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611135    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" containerName="maas-api"
Sep 19 09:27:21.611423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.611141    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="49f6759a-f9af-4c27-bef3-e523dc5b50b6" containerName="payload-processing"
Sep 19 09:27:21.615434 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.615413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.617692 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.617675    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0e596ff1-openshift-default-dockercfg-k5ctf\""
Sep 19 09:27:21.627885 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.627862    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"]
Sep 19 09:27:21.634244 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.634222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:21.672322 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-podinfo\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672493 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-credential-socket\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672493 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672367    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-token\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672493 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2xcn\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-kube-api-access-l2xcn\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672493 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-envoy\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672522    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-data\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-socket\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672609    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istiod-ca-cert\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.672719 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.672631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-certs\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773429 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istiod-ca-cert\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773429 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773429    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-certs\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-podinfo\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773611    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-credential-socket\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773629    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-token\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2xcn\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-kube-api-access-l2xcn\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-envoy\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-data\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773781    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-socket\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.773953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.773788    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-certs\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.774128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.774006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-credential-socket\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.774197 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.774179    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istiod-ca-cert\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.774339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.774319    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-data\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.774395 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.774346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-socket\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.776181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.776159    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-envoy\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.776501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.776479    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-podinfo\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.782245 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.782221    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-token\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.782374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.782350    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2xcn\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-kube-api-access-l2xcn\") pod \"e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") " pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.926309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.926278    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:21.941424 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:21.941390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:22.048512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.048478    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:22.207944 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.207920    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"]
Sep 19 09:27:22.209657 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:27:22.209624    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod152abdcd_512c_4de9_9b5f_9c68741bf6f0.slice/crio-ff668dda8c3a6a17705add57ca3e17df5e7f4290be28b10764ffb1784d4c72cd WatchSource:0}: Error finding container ff668dda8c3a6a17705add57ca3e17df5e7f4290be28b10764ffb1784d4c72cd: Status 404 returned error can't find the container with id ff668dda8c3a6a17705add57ca3e17df5e7f4290be28b10764ffb1784d4c72cd
Sep 19 09:27:22.212044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.212010    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:22.212158 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.212080    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:22.212158 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.212146    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:22.651751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.651712    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:22.836422 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.832944    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49f6759a-f9af-4c27-bef3-e523dc5b50b6" path="/var/lib/kubelet/pods/49f6759a-f9af-4c27-bef3-e523dc5b50b6/volumes"
Sep 19 09:27:22.836422 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.833567    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86f4375a-ca06-4c3c-ba5c-9a94488fa3dc" path="/var/lib/kubelet/pods/86f4375a-ca06-4c3c-ba5c-9a94488fa3dc/volumes"
Sep 19 09:27:22.836422 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:22.834057    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8d5218-cf14-4785-8e2f-55e2c1bb68ae" path="/var/lib/kubelet/pods/bb8d5218-cf14-4785-8e2f-55e2c1bb68ae/volumes"
Sep 19 09:27:23.065798 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.065711    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" event={"ID":"152abdcd-512c-4de9-9b5f-9c68741bf6f0","Type":"ContainerStarted","Data":"2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8"}
Sep 19 09:27:23.065798 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.065761    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" event={"ID":"152abdcd-512c-4de9-9b5f-9c68741bf6f0","Type":"ContainerStarted","Data":"ff668dda8c3a6a17705add57ca3e17df5e7f4290be28b10764ffb1784d4c72cd"}
Sep 19 09:27:23.088806 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.088751    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" podStartSLOduration=2.088733724 podStartE2EDuration="2.088733724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:23.086749723 +0000 UTC m=+1256.773555025" watchObservedRunningTime="2026-09-19 09:27:23.088733724 +0000 UTC m=+1256.775539021"
Sep 19 09:27:23.557168 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.557132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:23.680533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.680490    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:23.680964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.680548    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:23.693235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.693205    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:23.926666 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.926624    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:23.932063 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:23.932033    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:24.071539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.071494    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:24.072494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.072465    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:24.082492 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.082466    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:27:24.407488 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.407453    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:24.516768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.516736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:24.608259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.608220    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:24.689596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.689506    2581 patch_prober.go:28] interesting pod/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 19 09:27:24.689596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:24.689578    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 09:27:25.310834 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.310795    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:25.979003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.978965    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"]
Sep 19 09:27:25.982915 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.982890    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:25.985387 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.985361    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0e596ff1-serving-cert\""
Sep 19 09:27:25.985552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.985386    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 09:27:25.985552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.985462    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzltl\""
Sep 19 09:27:25.986309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.986290    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 09:27:25.986429 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.986326    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:27:25.996297 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:25.996271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"]
Sep 19 09:27:26.014864 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.014830    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:26.078712 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.078689    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:27:26.081703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.081640    2581 generic.go:356] "Generic (PLEG): container finished" podID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerID="c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574" exitCode=0
Sep 19 09:27:26.081703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.081690    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"
Sep 19 09:27:26.081897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.081723    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" event={"ID":"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769","Type":"ContainerDied","Data":"c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574"}
Sep 19 09:27:26.081897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.081768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4" event={"ID":"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769","Type":"ContainerDied","Data":"4f1872261438a293b53402e171e32ef2403e5ba4c37485454d63c8ced0480c73"}
Sep 19 09:27:26.081897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.081792    2581 scope.go:117] "RemoveContainer" containerID="c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574"
Sep 19 09:27:26.090725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.090697    2581 scope.go:117] "RemoveContainer" containerID="2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2"
Sep 19 09:27:26.100602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.100389    2581 scope.go:117] "RemoveContainer" containerID="c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574"
Sep 19 09:27:26.100830 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:26.100801    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574\": container with ID starting with c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574 not found: ID does not exist" containerID="c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574"
Sep 19 09:27:26.100930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.100838    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574"} err="failed to get container status \"c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574\": rpc error: code = NotFound desc = could not find container \"c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574\": container with ID starting with c8526fa79f53d06d4484141b09700939b29d732fa7cab4892480c3ce0be6a574 not found: ID does not exist"
Sep 19 09:27:26.100930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.100863    2581 scope.go:117] "RemoveContainer" containerID="2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2"
Sep 19 09:27:26.101347 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:26.101310    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2\": container with ID starting with 2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2 not found: ID does not exist" containerID="2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2"
Sep 19 09:27:26.101454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.101344    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2"} err="failed to get container status \"2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2\": rpc error: code = NotFound desc = could not find container \"2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2\": container with ID starting with 2abdf3e842549075011100a29f7e48dc51d7f147a2cf4c385cf34c502e16fac2 not found: ID does not exist"
Sep 19 09:27:26.106600 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.106565    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:26.117959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.117930    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp5jb\" (UniqueName: \"kubernetes.io/projected/45401ff6-5e3e-4c98-892b-cf4669140b23-kube-api-access-pp5jb\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.118135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.117965    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45401ff6-5e3e-4c98-892b-cf4669140b23-openshift-service-ca\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.118135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.118015    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-maas-api-tls\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.118135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.118067    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-metrics-certs\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.218692 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218657    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-data\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.218692 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218690    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-podinfo\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.218929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218713    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-credential-socket\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.218929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218734    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-envoy\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.218929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lnmt\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-kube-api-access-5lnmt\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.218929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218884    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-certs\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.219175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218973    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-token\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.219175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219017    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-socket\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.219175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219049    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istiod-ca-cert\") pod \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\" (UID: \"f50f4adf-6d6b-4498-9e5a-45c6c8d2b769\") "
Sep 19 09:27:26.219175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218967    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:27:26.219175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.218985    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-data" (OuterVolumeSpecName: "istio-data") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:27:26.219414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219212    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:27:26.219414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219329    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp5jb\" (UniqueName: \"kubernetes.io/projected/45401ff6-5e3e-4c98-892b-cf4669140b23-kube-api-access-pp5jb\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.219414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219344    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:27:26.219414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45401ff6-5e3e-4c98-892b-cf4669140b23-openshift-service-ca\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.219606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219417    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-maas-api-tls\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.219606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.219538    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-metrics-certs\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.220488 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.220047    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:27:26.220742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.220662    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-data\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.220742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.220691    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-credential-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.220742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.220707    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.220742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.220720    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-workload-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.221347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.221322    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45401ff6-5e3e-4c98-892b-cf4669140b23-openshift-service-ca\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.221865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.221837    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:27:26.221973 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.221867    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-token" (OuterVolumeSpecName: "istio-token") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:26.222192 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.222156    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 09:27:26.222437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.222416    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-kube-api-access-5lnmt" (OuterVolumeSpecName: "kube-api-access-5lnmt") pod "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" (UID: "f50f4adf-6d6b-4498-9e5a-45c6c8d2b769"). InnerVolumeSpecName "kube-api-access-5lnmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:26.222483 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.222456    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-metrics-certs\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.222974 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.222960    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-maas-api-tls\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.227360 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.227339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp5jb\" (UniqueName: \"kubernetes.io/projected/45401ff6-5e3e-4c98-892b-cf4669140b23-kube-api-access-pp5jb\") pod \"maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.253321 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.253291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:26.296245 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.296218    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:26.322033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.322005    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-envoy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.322033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.322035    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lnmt\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-kube-api-access-5lnmt\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.322217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.322049    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.322217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.322063    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istiod-ca-cert\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.322217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.322073    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769-istio-podinfo\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:26.407916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.407876    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"]
Sep 19 09:27:26.409617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.409596    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d6a0ebf6-gw-openshift-default-57bfbb786d-q74k4"]
Sep 19 09:27:26.512207 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.512172    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:26.572332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.572310    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"]
Sep 19 09:27:26.574577 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:27:26.574548    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45401ff6_5e3e_4c98_892b_cf4669140b23.slice/crio-b6f8a5f8a14bfb1366bd6c6e630338313d9fe5b49c63623fd7848d3058233178 WatchSource:0}: Error finding container b6f8a5f8a14bfb1366bd6c6e630338313d9fe5b49c63623fd7848d3058233178: Status 404 returned error can't find the container with id b6f8a5f8a14bfb1366bd6c6e630338313d9fe5b49c63623fd7848d3058233178
Sep 19 09:27:26.610339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.610251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:26.633928 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.633893    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"]
Sep 19 09:27:26.634253 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.634241    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy"
Sep 19 09:27:26.634296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.634256    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy"
Sep 19 09:27:26.634296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.634266    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy"
Sep 19 09:27:26.634296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.634271    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy"
Sep 19 09:27:26.634386 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.634339    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy"
Sep 19 09:27:26.638501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.638485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.643144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.643087    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0e596ff1-dockercfg-kq4vt\""
Sep 19 09:27:26.643397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.643384    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0e596ff1\""
Sep 19 09:27:26.652250 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.650257    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"]
Sep 19 09:27:26.690813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.690779    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"]
Sep 19 09:27:26.691230 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.691217    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" containerName="istio-proxy"
Sep 19 09:27:26.694221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.694206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:26.704623 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.704589    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"]
Sep 19 09:27:26.726072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.726043    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxcc8\" (UniqueName: \"kubernetes.io/projected/fcdfd1f5-1203-426a-8371-dab002d76551-kube-api-access-lxcc8\") pod \"payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") " pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.726237 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.726081    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcdfd1f5-1203-426a-8371-dab002d76551-plugins-config-volume\") pod \"payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") " pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.827047 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.827008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9s42\" (UniqueName: \"kubernetes.io/projected/e5e9978b-5b1b-4e90-931a-7ead466a8f28-kube-api-access-k9s42\") pod \"payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:26.827236 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.827068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxcc8\" (UniqueName: \"kubernetes.io/projected/fcdfd1f5-1203-426a-8371-dab002d76551-kube-api-access-lxcc8\") pod \"payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") " pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.827236 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.827142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcdfd1f5-1203-426a-8371-dab002d76551-plugins-config-volume\") pod \"payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") " pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.827236 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.827179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9978b-5b1b-4e90-931a-7ead466a8f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:26.829440 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.829408    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0e596ff1\""
Sep 19 09:27:26.830750 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.830728    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50f4adf-6d6b-4498-9e5a-45c6c8d2b769" path="/var/lib/kubelet/pods/f50f4adf-6d6b-4498-9e5a-45c6c8d2b769/volumes"
Sep 19 09:27:26.835338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.835312    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxcc8\" (UniqueName: \"kubernetes.io/projected/fcdfd1f5-1203-426a-8371-dab002d76551-kube-api-access-lxcc8\") pod \"payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") " pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.838581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.838556    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcdfd1f5-1203-426a-8371-dab002d76551-plugins-config-volume\") pod \"payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") " pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.928488 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.928454    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9978b-5b1b-4e90-931a-7ead466a8f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:26.928661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.928557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9s42\" (UniqueName: \"kubernetes.io/projected/e5e9978b-5b1b-4e90-931a-7ead466a8f28-kube-api-access-k9s42\") pod \"payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:26.929211 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.929193    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9978b-5b1b-4e90-931a-7ead466a8f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:26.936637 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.936608    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9s42\" (UniqueName: \"kubernetes.io/projected/e5e9978b-5b1b-4e90-931a-7ead466a8f28-kube-api-access-k9s42\") pod \"payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:26.951477 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.951454    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0e596ff1-dockercfg-kq4vt\""
Sep 19 09:27:26.959338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.959306    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:26.993010 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:26.992981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:27.005227 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.005201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:27.089777 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.089739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87" event={"ID":"45401ff6-5e3e-4c98-892b-cf4669140b23","Type":"ContainerStarted","Data":"083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b"}
Sep 19 09:27:27.089905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.089805    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87" event={"ID":"45401ff6-5e3e-4c98-892b-cf4669140b23","Type":"ContainerStarted","Data":"b6f8a5f8a14bfb1366bd6c6e630338313d9fe5b49c63623fd7848d3058233178"}
Sep 19 09:27:27.089905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.089839    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:27.106712 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.106645    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87" podStartSLOduration=1.7065620099999999 podStartE2EDuration="2.106621904s" totalImagesPullingTime="400.059894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:27.106079077 +0000 UTC m=+1260.792884376" watchObservedRunningTime="2026-09-19 09:27:27.106621904 +0000 UTC m=+1260.793427203"
Sep 19 09:27:27.251904 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.251878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"]
Sep 19 09:27:27.253986 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:27:27.253946    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcdfd1f5_1203_426a_8371_dab002d76551.slice/crio-ad5210547655eee88021092e7215c8858814021e60c332605c496167c9512f65 WatchSource:0}: Error finding container ad5210547655eee88021092e7215c8858814021e60c332605c496167c9512f65: Status 404 returned error can't find the container with id ad5210547655eee88021092e7215c8858814021e60c332605c496167c9512f65
Sep 19 09:27:27.291736 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.291701    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"]
Sep 19 09:27:27.301238 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:27:27.301206    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e9978b_5b1b_4e90_931a_7ead466a8f28.slice/crio-823e24cc00d9e4c7ec0944ec010f701b180f21d16619048d95434a90c8303183 WatchSource:0}: Error finding container 823e24cc00d9e4c7ec0944ec010f701b180f21d16619048d95434a90c8303183: Status 404 returned error can't find the container with id 823e24cc00d9e4c7ec0944ec010f701b180f21d16619048d95434a90c8303183
Sep 19 09:27:27.673837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:27.673798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:28.097749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.097650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j" event={"ID":"e5e9978b-5b1b-4e90-931a-7ead466a8f28","Type":"ContainerStarted","Data":"b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956"}
Sep 19 09:27:28.097749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.097718    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:28.097749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.097749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j" event={"ID":"e5e9978b-5b1b-4e90-931a-7ead466a8f28","Type":"ContainerStarted","Data":"823e24cc00d9e4c7ec0944ec010f701b180f21d16619048d95434a90c8303183"}
Sep 19 09:27:28.099462 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.099434    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm" event={"ID":"fcdfd1f5-1203-426a-8371-dab002d76551","Type":"ContainerStarted","Data":"7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad"}
Sep 19 09:27:28.099587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.099488    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm" event={"ID":"fcdfd1f5-1203-426a-8371-dab002d76551","Type":"ContainerStarted","Data":"ad5210547655eee88021092e7215c8858814021e60c332605c496167c9512f65"}
Sep 19 09:27:28.118470 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.118420    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j" podStartSLOduration=2.118398684 podStartE2EDuration="2.118398684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:28.116584016 +0000 UTC m=+1261.803389331" watchObservedRunningTime="2026-09-19 09:27:28.118398684 +0000 UTC m=+1261.805204036"
Sep 19 09:27:28.136686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.136621    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm" podStartSLOduration=2.136601522 podStartE2EDuration="2.136601522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:28.135852953 +0000 UTC m=+1261.822658252" watchObservedRunningTime="2026-09-19 09:27:28.136601522 +0000 UTC m=+1261.823406821"
Sep 19 09:27:28.453897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:28.453858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:29.104763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:29.104733    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:29.293895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:29.293858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:29.995665 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:29.995624    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:33.105607 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:33.105573    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:35.156110 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.156075    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"]
Sep 19 09:27:35.159386 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.159364    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.162970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.162939    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-0ce808\"/\"openshift-service-ca.crt\""
Sep 19 09:27:35.162970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.162958    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-0ce808\"/\"default-dockercfg-znvmt\""
Sep 19 09:27:35.163201 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.162958    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-0ce808\"/\"kube-root-ca.crt\""
Sep 19 09:27:35.163201 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.162967    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-0ce808\"/\"autores-model-0ce808-kserve-self-signed-certs\""
Sep 19 09:27:35.169718 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.169695    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"]
Sep 19 09:27:35.320413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.320377    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-dshm\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.320594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.320499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60ec054a-f2e6-4f13-863f-a9cb183211bb-tls-certs\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.320594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.320549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-tmp-dir\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.320594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.320582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-model-cache\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.320768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.320607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-home\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.320768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.320625    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2kw8\" (UniqueName: \"kubernetes.io/projected/60ec054a-f2e6-4f13-863f-a9cb183211bb-kube-api-access-x2kw8\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.359518 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.359480    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:35.421717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.421616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-dshm\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.421876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.421720    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60ec054a-f2e6-4f13-863f-a9cb183211bb-tls-certs\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.421876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.421758    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-tmp-dir\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.421876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.421801    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-model-cache\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.421876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.421835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-home\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.421876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.421859    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2kw8\" (UniqueName: \"kubernetes.io/projected/60ec054a-f2e6-4f13-863f-a9cb183211bb-kube-api-access-x2kw8\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.422252 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.422226    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-model-cache\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.422367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.422308    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-home\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.422367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.422347    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-tmp-dir\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.424130 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.424094    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-dshm\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.424459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.424440    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60ec054a-f2e6-4f13-863f-a9cb183211bb-tls-certs\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.431832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.431809    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2kw8\" (UniqueName: \"kubernetes.io/projected/60ec054a-f2e6-4f13-863f-a9cb183211bb-kube-api-access-x2kw8\") pod \"autores-model-0ce808-kserve-6f5fcf77c9-f5sdb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") " pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.469895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.469861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:35.769389 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:35.769363    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"]
Sep 19 09:27:35.771350 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:27:35.771321    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60ec054a_f2e6_4f13_863f_a9cb183211bb.slice/crio-8006813ac3e5516766df6e5b965f74a08ceb3c75c8440a91eeb2e7fa7fda0516 WatchSource:0}: Error finding container 8006813ac3e5516766df6e5b965f74a08ceb3c75c8440a91eeb2e7fa7fda0516: Status 404 returned error can't find the container with id 8006813ac3e5516766df6e5b965f74a08ceb3c75c8440a91eeb2e7fa7fda0516
Sep 19 09:27:36.143246 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:36.143211    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb" event={"ID":"60ec054a-f2e6-4f13-863f-a9cb183211bb","Type":"ContainerStarted","Data":"88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6"}
Sep 19 09:27:36.143246 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:36.143250    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb" event={"ID":"60ec054a-f2e6-4f13-863f-a9cb183211bb","Type":"ContainerStarted","Data":"8006813ac3e5516766df6e5b965f74a08ceb3c75c8440a91eeb2e7fa7fda0516"}
Sep 19 09:27:36.161932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:36.161877    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb" podStartSLOduration=1.161859819 podStartE2EDuration="1.161859819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:36.160160026 +0000 UTC m=+1269.846965322" watchObservedRunningTime="2026-09-19 09:27:36.161859819 +0000 UTC m=+1269.848665116"
Sep 19 09:27:39.107750 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:39.106797    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:40.115719 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:40.115684    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:43.079554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:43.079518    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:27:44.184266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:44.184228    2581 generic.go:356] "Generic (PLEG): container finished" podID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerID="2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8" exitCode=0
Sep 19 09:27:44.184709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:44.184304    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" event={"ID":"152abdcd-512c-4de9-9b5f-9c68741bf6f0","Type":"ContainerDied","Data":"2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8"}
Sep 19 09:27:44.184709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:44.184692    2581 scope.go:117] "RemoveContainer" containerID="2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8"
Sep 19 09:27:44.185921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:44.185894    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:44.185989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:44.185965    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:44.186027 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:44.185996    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:27:45.195641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:45.195598    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" event={"ID":"152abdcd-512c-4de9-9b5f-9c68741bf6f0","Type":"ContainerStarted","Data":"782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b"}
Sep 19 09:27:45.470717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:45.470622    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:45.470717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:45.470696    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:45.483030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:45.483010    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:45.927301 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:45.927257    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:45.927552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:45.927527    2581 patch_prober.go:28] interesting pod/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 09:27:45.927663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:45.927583    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 09:27:46.220860 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:46.220779    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:27:46.926670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:46.926633    2581 patch_prober.go:28] interesting pod/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 09:27:46.926871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:46.926687    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 09:27:47.927655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:47.927611    2581 patch_prober.go:28] interesting pod/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 09:27:47.928187 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:47.927693    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 09:27:48.066141 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:48.066077    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:48.927237 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:48.927200    2581 patch_prober.go:28] interesting pod/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 09:27:48.927428 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:48.927258    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 09:27:49.010517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.010481    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:49.570629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.570590    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"]
Sep 19 09:27:49.574030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.574013    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.578274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.577511    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9dfb0f\"/\"kube-root-ca.crt\""
Sep 19 09:27:49.578274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.578030    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9dfb0f\"/\"test-model-9dfb0f-kserve-self-signed-certs\""
Sep 19 09:27:49.578274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.578031    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9dfb0f\"/\"default-dockercfg-bcgfp\""
Sep 19 09:27:49.578274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.578247    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9dfb0f\"/\"openshift-service-ca.crt\""
Sep 19 09:27:49.608213 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.608154    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"]
Sep 19 09:27:49.653424 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.653389    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-home\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.653606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.653435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-dshm\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.653606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.653461    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-tmp-dir\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.653606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.653482    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84910cf6-750e-4958-90fe-3a2febec319f-tls-certs\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.653606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.653535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7prmb\" (UniqueName: \"kubernetes.io/projected/84910cf6-750e-4958-90fe-3a2febec319f-kube-api-access-7prmb\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.653606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.653576    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-model-cache\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.702515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.702481    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:49.754988 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.754953    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7prmb\" (UniqueName: \"kubernetes.io/projected/84910cf6-750e-4958-90fe-3a2febec319f-kube-api-access-7prmb\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-model-cache\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755076    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-home\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-dshm\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-tmp-dir\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755187    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84910cf6-750e-4958-90fe-3a2febec319f-tls-certs\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755413    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-model-cache\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755545    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-home\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.755639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.755618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-tmp-dir\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.757510 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.757488    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-dshm\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.757962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.757942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84910cf6-750e-4958-90fe-3a2febec319f-tls-certs\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.765607 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.765574    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7prmb\" (UniqueName: \"kubernetes.io/projected/84910cf6-750e-4958-90fe-3a2febec319f-kube-api-access-7prmb\") pod \"test-model-9dfb0f-kserve-5f5d8f6875-h4sfn\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") " pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.895751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.895707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:49.926725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.926684    2581 patch_prober.go:28] interesting pod/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Sep 19 09:27:49.926872 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:49.926773    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Sep 19 09:27:50.002642 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:50.001637    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:50.183730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:50.183699    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"]
Sep 19 09:27:50.186797 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:27:50.186769    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84910cf6_750e_4958_90fe_3a2febec319f.slice/crio-f3e3134a30564a562e45dc2c8ade1bb150b46e7dbb79328fa1d80f90beacee0f WatchSource:0}: Error finding container f3e3134a30564a562e45dc2c8ade1bb150b46e7dbb79328fa1d80f90beacee0f: Status 404 returned error can't find the container with id f3e3134a30564a562e45dc2c8ade1bb150b46e7dbb79328fa1d80f90beacee0f
Sep 19 09:27:50.228676 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:50.228472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn" event={"ID":"84910cf6-750e-4958-90fe-3a2febec319f","Type":"ContainerStarted","Data":"f3e3134a30564a562e45dc2c8ade1bb150b46e7dbb79328fa1d80f90beacee0f"}
Sep 19 09:27:50.856466 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:50.856426    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:50.930952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:50.930928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:50.931345 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:50.931325    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:50.931948 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:50.931928    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:27:51.236306 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:51.236277    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn" event={"ID":"84910cf6-750e-4958-90fe-3a2febec319f","Type":"ContainerStarted","Data":"4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea"}
Sep 19 09:27:51.254525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:51.254478    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn" podStartSLOduration=2.254462099 podStartE2EDuration="2.254462099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:27:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:27:51.252783942 +0000 UTC m=+1284.939589240" watchObservedRunningTime="2026-09-19 09:27:51.254462099 +0000 UTC m=+1284.941267398"
Sep 19 09:27:51.581225 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:51.581138    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:52.410932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:52.410898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:53.121189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:53.121152    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:53.977493 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:53.977455    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:54.728127 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:54.728079    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:55.581819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:55.581733    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:56.173554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.173520    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"]
Sep 19 09:27:56.173772 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.173725    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87" podUID="45401ff6-5e3e-4c98-892b-cf4669140b23" containerName="maas-api" containerID="cri-o://083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b" gracePeriod=30
Sep 19 09:27:56.205247 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.205197    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:56.244372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.244338    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"]
Sep 19 09:27:56.244629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.244567    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm" podUID="fcdfd1f5-1203-426a-8371-dab002d76551" containerName="payload-processing" containerID="cri-o://7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad" gracePeriod=30
Sep 19 09:27:56.250933 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.250825    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"]
Sep 19 09:27:56.251490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.251460    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j" podUID="e5e9978b-5b1b-4e90-931a-7ead466a8f28" containerName="payload-pre-processing" containerID="cri-o://b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956" gracePeriod=30
Sep 19 09:27:56.287278 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.287231    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:56.357241 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.357203    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:56.451571 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.451533    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:56.506671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.506642    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:56.509776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.509755    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:56.533650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.533606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:56.538771 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.538748    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:56.622794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.622765    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-metrics-certs\") pod \"45401ff6-5e3e-4c98-892b-cf4669140b23\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.622807    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45401ff6-5e3e-4c98-892b-cf4669140b23-openshift-service-ca\") pod \"45401ff6-5e3e-4c98-892b-cf4669140b23\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.622854    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcdfd1f5-1203-426a-8371-dab002d76551-plugins-config-volume\") pod \"fcdfd1f5-1203-426a-8371-dab002d76551\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.622897    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-maas-api-tls\") pod \"45401ff6-5e3e-4c98-892b-cf4669140b23\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.622957    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxcc8\" (UniqueName: \"kubernetes.io/projected/fcdfd1f5-1203-426a-8371-dab002d76551-kube-api-access-lxcc8\") pod \"fcdfd1f5-1203-426a-8371-dab002d76551\" (UID: \"fcdfd1f5-1203-426a-8371-dab002d76551\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.622990    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9s42\" (UniqueName: \"kubernetes.io/projected/e5e9978b-5b1b-4e90-931a-7ead466a8f28-kube-api-access-k9s42\") pod \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.623042    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9978b-5b1b-4e90-931a-7ead466a8f28-plugins-config-volume\") pod \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\" (UID: \"e5e9978b-5b1b-4e90-931a-7ead466a8f28\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.623068    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp5jb\" (UniqueName: \"kubernetes.io/projected/45401ff6-5e3e-4c98-892b-cf4669140b23-kube-api-access-pp5jb\") pod \"45401ff6-5e3e-4c98-892b-cf4669140b23\" (UID: \"45401ff6-5e3e-4c98-892b-cf4669140b23\") "
Sep 19 09:27:56.623267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.623256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45401ff6-5e3e-4c98-892b-cf4669140b23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45401ff6-5e3e-4c98-892b-cf4669140b23" (UID: "45401ff6-5e3e-4c98-892b-cf4669140b23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:27:56.623713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.623294    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcdfd1f5-1203-426a-8371-dab002d76551-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fcdfd1f5-1203-426a-8371-dab002d76551" (UID: "fcdfd1f5-1203-426a-8371-dab002d76551"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:27:56.623713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.623559    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45401ff6-5e3e-4c98-892b-cf4669140b23-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:56.623713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.623646    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcdfd1f5-1203-426a-8371-dab002d76551-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:56.623713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.623694    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5e9978b-5b1b-4e90-931a-7ead466a8f28-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5e9978b-5b1b-4e90-931a-7ead466a8f28" (UID: "e5e9978b-5b1b-4e90-931a-7ead466a8f28"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:27:56.625491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.625462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "45401ff6-5e3e-4c98-892b-cf4669140b23" (UID: "45401ff6-5e3e-4c98-892b-cf4669140b23"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:27:56.625694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.625673    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45401ff6-5e3e-4c98-892b-cf4669140b23-kube-api-access-pp5jb" (OuterVolumeSpecName: "kube-api-access-pp5jb") pod "45401ff6-5e3e-4c98-892b-cf4669140b23" (UID: "45401ff6-5e3e-4c98-892b-cf4669140b23"). InnerVolumeSpecName "kube-api-access-pp5jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:56.625790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.625772    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcdfd1f5-1203-426a-8371-dab002d76551-kube-api-access-lxcc8" (OuterVolumeSpecName: "kube-api-access-lxcc8") pod "fcdfd1f5-1203-426a-8371-dab002d76551" (UID: "fcdfd1f5-1203-426a-8371-dab002d76551"). InnerVolumeSpecName "kube-api-access-lxcc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:56.625840 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.625774    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45401ff6-5e3e-4c98-892b-cf4669140b23" (UID: "45401ff6-5e3e-4c98-892b-cf4669140b23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:27:56.625877 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.625834    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5e9978b-5b1b-4e90-931a-7ead466a8f28-kube-api-access-k9s42" (OuterVolumeSpecName: "kube-api-access-k9s42") pod "e5e9978b-5b1b-4e90-931a-7ead466a8f28" (UID: "e5e9978b-5b1b-4e90-931a-7ead466a8f28"). InnerVolumeSpecName "kube-api-access-k9s42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:27:56.725008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.724897    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5e9978b-5b1b-4e90-931a-7ead466a8f28-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:56.725008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.724945    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp5jb\" (UniqueName: \"kubernetes.io/projected/45401ff6-5e3e-4c98-892b-cf4669140b23-kube-api-access-pp5jb\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:56.725008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.724956    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:56.725008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.724966    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45401ff6-5e3e-4c98-892b-cf4669140b23-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:56.725008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.724975    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxcc8\" (UniqueName: \"kubernetes.io/projected/fcdfd1f5-1203-426a-8371-dab002d76551-kube-api-access-lxcc8\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:56.725008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:56.724983    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9s42\" (UniqueName: \"kubernetes.io/projected/e5e9978b-5b1b-4e90-931a-7ead466a8f28-kube-api-access-k9s42\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:27:57.272521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.272482    2581 generic.go:356] "Generic (PLEG): container finished" podID="e5e9978b-5b1b-4e90-931a-7ead466a8f28" containerID="b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956" exitCode=0
Sep 19 09:27:57.272725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.272527    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"
Sep 19 09:27:57.272725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.272571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j" event={"ID":"e5e9978b-5b1b-4e90-931a-7ead466a8f28","Type":"ContainerDied","Data":"b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956"}
Sep 19 09:27:57.272725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.272632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j" event={"ID":"e5e9978b-5b1b-4e90-931a-7ead466a8f28","Type":"ContainerDied","Data":"823e24cc00d9e4c7ec0944ec010f701b180f21d16619048d95434a90c8303183"}
Sep 19 09:27:57.272725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.272664    2581 scope.go:117] "RemoveContainer" containerID="b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956"
Sep 19 09:27:57.274177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.274094    2581 generic.go:356] "Generic (PLEG): container finished" podID="45401ff6-5e3e-4c98-892b-cf4669140b23" containerID="083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b" exitCode=0
Sep 19 09:27:57.274177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.274155    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87" event={"ID":"45401ff6-5e3e-4c98-892b-cf4669140b23","Type":"ContainerDied","Data":"083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b"}
Sep 19 09:27:57.274316 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.274232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87" event={"ID":"45401ff6-5e3e-4c98-892b-cf4669140b23","Type":"ContainerDied","Data":"b6f8a5f8a14bfb1366bd6c6e630338313d9fe5b49c63623fd7848d3058233178"}
Sep 19 09:27:57.274316 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.274163    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"
Sep 19 09:27:57.275763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.275736    2581 generic.go:356] "Generic (PLEG): container finished" podID="fcdfd1f5-1203-426a-8371-dab002d76551" containerID="7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad" exitCode=0
Sep 19 09:27:57.275882 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.275777    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"
Sep 19 09:27:57.275882 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.275782    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm" event={"ID":"fcdfd1f5-1203-426a-8371-dab002d76551","Type":"ContainerDied","Data":"7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad"}
Sep 19 09:27:57.275882 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.275824    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm" event={"ID":"fcdfd1f5-1203-426a-8371-dab002d76551","Type":"ContainerDied","Data":"ad5210547655eee88021092e7215c8858814021e60c332605c496167c9512f65"}
Sep 19 09:27:57.282886 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.282855    2581 scope.go:117] "RemoveContainer" containerID="b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956"
Sep 19 09:27:57.283281 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:57.283254    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956\": container with ID starting with b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956 not found: ID does not exist" containerID="b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956"
Sep 19 09:27:57.283354 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.283289    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956"} err="failed to get container status \"b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956\": rpc error: code = NotFound desc = could not find container \"b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956\": container with ID starting with b7e97d7128ad3d4a364ede7d1ef78784be574deac7e69563f7c63e4b78954956 not found: ID does not exist"
Sep 19 09:27:57.283354 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.283307    2581 scope.go:117] "RemoveContainer" containerID="083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b"
Sep 19 09:27:57.291873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.291854    2581 scope.go:117] "RemoveContainer" containerID="083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b"
Sep 19 09:27:57.292212 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:57.292185    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b\": container with ID starting with 083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b not found: ID does not exist" containerID="083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b"
Sep 19 09:27:57.292284 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.292221    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b"} err="failed to get container status \"083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b\": rpc error: code = NotFound desc = could not find container \"083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b\": container with ID starting with 083e3184e284fcb7a3005873ef9ce825f0a1c05ced45aa030c7eb8ca1bab594b not found: ID does not exist"
Sep 19 09:27:57.292284 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.292238    2581 scope.go:117] "RemoveContainer" containerID="7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad"
Sep 19 09:27:57.296358 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.296332    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"]
Sep 19 09:27:57.298915 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.298885    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:57.300052 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.300031    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0e596ff1-6994d5b9c6-qkb87"]
Sep 19 09:27:57.302148 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.302132    2581 scope.go:117] "RemoveContainer" containerID="7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad"
Sep 19 09:27:57.302448 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:27:57.302431    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad\": container with ID starting with 7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad not found: ID does not exist" containerID="7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad"
Sep 19 09:27:57.302489 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.302458    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad"} err="failed to get container status \"7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad\": rpc error: code = NotFound desc = could not find container \"7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad\": container with ID starting with 7b9c7ad9a8fa78eda4c713160311721135165aaf71a5cf8d09cba9c1e71263ad not found: ID does not exist"
Sep 19 09:27:57.308816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.308792    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"]
Sep 19 09:27:57.315463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.315432    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0e596ff1-6d9469ccb7-658dm"]
Sep 19 09:27:57.326691 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.326664    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"]
Sep 19 09:27:57.331694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.331668    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0e596ff1-6688bc95b6-89n2j"]
Sep 19 09:27:57.603802 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.603712    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"]
Sep 19 09:27:57.603954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.603918    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy" containerID="cri-o://782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b" gracePeriod=30
Sep 19 09:27:57.651539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:57.651499    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:58.075552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:58.075521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:58.307408 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:58.307364    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:58.411703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:58.411664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:58.704147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:58.704046    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:58.833115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:58.833067    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45401ff6-5e3e-4c98-892b-cf4669140b23" path="/var/lib/kubelet/pods/45401ff6-5e3e-4c98-892b-cf4669140b23/volumes"
Sep 19 09:27:58.833499 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:58.833484    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5e9978b-5b1b-4e90-931a-7ead466a8f28" path="/var/lib/kubelet/pods/e5e9978b-5b1b-4e90-931a-7ead466a8f28/volumes"
Sep 19 09:27:58.833799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:58.833788    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcdfd1f5-1203-426a-8371-dab002d76551" path="/var/lib/kubelet/pods/fcdfd1f5-1203-426a-8371-dab002d76551/volumes"
Sep 19 09:27:59.152303 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:59.152265    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:27:59.896605 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:59.896569    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:59.897070 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:59.896622    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:27:59.909149 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:27:59.909122    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:28:00.111312 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:00.111282    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:00.305739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:00.305653    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:28:00.867679 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:00.867641    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:01.164878 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:01.164843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:01.330601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:01.330570    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:28:01.352714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:01.352683    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:02.241814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.241779    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:02.859455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.859434    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:28:02.984925 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.984888    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-data\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.984979    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-envoy\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985006    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istiod-ca-cert\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985036    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-token\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985087    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-credential-socket\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985151    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-podinfo\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985187    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-socket\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985218    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2xcn\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-kube-api-access-l2xcn\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985253    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-certs\") pod \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\" (UID: \"152abdcd-512c-4de9-9b5f-9c68741bf6f0\") "
Sep 19 09:28:02.985404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985256    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:02.985622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985527    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:02.985677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985626    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:02.985677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985674    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-data\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:02.985775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985711    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:02.985867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985814    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:02.986057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.985872    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:28:02.987670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.987637    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:28:02.987670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.987652    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:02.987852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.987816    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-kube-api-access-l2xcn" (OuterVolumeSpecName: "kube-api-access-l2xcn") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "kube-api-access-l2xcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:28:02.988008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:02.987987    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "152abdcd-512c-4de9-9b5f-9c68741bf6f0" (UID: "152abdcd-512c-4de9-9b5f-9c68741bf6f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 09:28:03.086509 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.086471    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2xcn\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-kube-api-access-l2xcn\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:03.086509 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.086501    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-workload-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:03.086509 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.086512    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-envoy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:03.086509 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.086520    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istiod-ca-cert\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:03.086783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.086527    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:03.086783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.086535    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/152abdcd-512c-4de9-9b5f-9c68741bf6f0-credential-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:03.086783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.086549    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/152abdcd-512c-4de9-9b5f-9c68741bf6f0-istio-podinfo\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:03.181203 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181169    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"]
Sep 19 09:28:03.181508 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181496    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy"
Sep 19 09:28:03.181559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181510    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy"
Sep 19 09:28:03.181559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181525    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45401ff6-5e3e-4c98-892b-cf4669140b23" containerName="maas-api"
Sep 19 09:28:03.181559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181530    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="45401ff6-5e3e-4c98-892b-cf4669140b23" containerName="maas-api"
Sep 19 09:28:03.181559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181541    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcdfd1f5-1203-426a-8371-dab002d76551" containerName="payload-processing"
Sep 19 09:28:03.181559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181547    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcdfd1f5-1203-426a-8371-dab002d76551" containerName="payload-processing"
Sep 19 09:28:03.181559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181554    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e9978b-5b1b-4e90-931a-7ead466a8f28" containerName="payload-pre-processing"
Sep 19 09:28:03.181559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181559    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e9978b-5b1b-4e90-931a-7ead466a8f28" containerName="payload-pre-processing"
Sep 19 09:28:03.181764 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181622    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="45401ff6-5e3e-4c98-892b-cf4669140b23" containerName="maas-api"
Sep 19 09:28:03.181764 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181629    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy"
Sep 19 09:28:03.181764 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181637    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5e9978b-5b1b-4e90-931a-7ead466a8f28" containerName="payload-pre-processing"
Sep 19 09:28:03.181764 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.181643    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcdfd1f5-1203-426a-8371-dab002d76551" containerName="payload-processing"
Sep 19 09:28:03.184876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.184849    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.187837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.187817    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 09:28:03.187988 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.187813    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-61e32580-serving-cert\""
Sep 19 09:28:03.187988 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.187861    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzltl\""
Sep 19 09:28:03.188199 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.188182    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:28:03.188259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.188219    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 09:28:03.196234 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.196208    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"]
Sep 19 09:28:03.216396 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.209751    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:03.288756 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.288658    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.288756 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.288691    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-metrics-certs\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.288756 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.288716    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc717f90-730f-4361-925d-510eae44dc5c-openshift-service-ca\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.289230 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.288809    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9llv\" (UniqueName: \"kubernetes.io/projected/dc717f90-730f-4361-925d-510eae44dc5c-kube-api-access-p9llv\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.307667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.307631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:03.312001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.311977    2581 generic.go:356] "Generic (PLEG): container finished" podID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerID="782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b" exitCode=0
Sep 19 09:28:03.312147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.312021    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" event={"ID":"152abdcd-512c-4de9-9b5f-9c68741bf6f0","Type":"ContainerDied","Data":"782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b"}
Sep 19 09:28:03.312147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.312041    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"
Sep 19 09:28:03.312147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.312110    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2" event={"ID":"152abdcd-512c-4de9-9b5f-9c68741bf6f0","Type":"ContainerDied","Data":"ff668dda8c3a6a17705add57ca3e17df5e7f4290be28b10764ffb1784d4c72cd"}
Sep 19 09:28:03.312147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.312128    2581 scope.go:117] "RemoveContainer" containerID="782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b"
Sep 19 09:28:03.321515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.321495    2581 scope.go:117] "RemoveContainer" containerID="2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8"
Sep 19 09:28:03.329984 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.329965    2581 scope.go:117] "RemoveContainer" containerID="782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b"
Sep 19 09:28:03.330301 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:03.330278    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b\": container with ID starting with 782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b not found: ID does not exist" containerID="782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b"
Sep 19 09:28:03.330404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.330314    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b"} err="failed to get container status \"782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b\": rpc error: code = NotFound desc = could not find container \"782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b\": container with ID starting with 782a5a42026b1b6719cc8bb6ae905ebe6c2940b4d8cf982db88d686e411b243b not found: ID does not exist"
Sep 19 09:28:03.330404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.330338    2581 scope.go:117] "RemoveContainer" containerID="2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8"
Sep 19 09:28:03.330616 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:03.330602    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8\": container with ID starting with 2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8 not found: ID does not exist" containerID="2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8"
Sep 19 09:28:03.330661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.330619    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8"} err="failed to get container status \"2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8\": rpc error: code = NotFound desc = could not find container \"2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8\": container with ID starting with 2d0522d8c007c133222ed4014fc65579120707b380a14a56f60c552827b762b8 not found: ID does not exist"
Sep 19 09:28:03.336675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.336651    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"]
Sep 19 09:28:03.340063 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.340040    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0e596ff1-openshift-default-668c7857b8-4sbz2"]
Sep 19 09:28:03.389560 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.389524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9llv\" (UniqueName: \"kubernetes.io/projected/dc717f90-730f-4361-925d-510eae44dc5c-kube-api-access-p9llv\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.389703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.389593    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.389703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.389618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-metrics-certs\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.389703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.389643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc717f90-730f-4361-925d-510eae44dc5c-openshift-service-ca\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.389894 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:03.389709    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-serving-cert: secret "maas-api-e2e-derive-61e32580-serving-cert" not found
Sep 19 09:28:03.389894 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:03.389803    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls podName:dc717f90-730f-4361-925d-510eae44dc5c nodeName:}" failed. No retries permitted until 2026-09-19 09:28:03.889779128 +0000 UTC m=+1297.576584418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls") pod "maas-api-e2e-derive-61e32580-76478fc64c-k86w9" (UID: "dc717f90-730f-4361-925d-510eae44dc5c") : secret "maas-api-e2e-derive-61e32580-serving-cert" not found
Sep 19 09:28:03.390416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.390346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc717f90-730f-4361-925d-510eae44dc5c-openshift-service-ca\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.392260 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.392240    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-metrics-certs\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.400255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.400229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9llv\" (UniqueName: \"kubernetes.io/projected/dc717f90-730f-4361-925d-510eae44dc5c-kube-api-access-p9llv\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.476265 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.476228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:03.699945 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.699909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:03.797543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.797509    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:03.834551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.834516    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"]
Sep 19 09:28:03.834904 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.834890    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy"
Sep 19 09:28:03.834950 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.834906    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy"
Sep 19 09:28:03.834992 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.834972    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" containerName="istio-proxy"
Sep 19 09:28:03.839272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.839255    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:03.841822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.841788    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-61e32580\""
Sep 19 09:28:03.841980 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.841798    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-61e32580-dockercfg-dzbmj\""
Sep 19 09:28:03.853447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.853421    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"]
Sep 19 09:28:03.894254 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.894221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.896854 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.896832    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls\") pod \"maas-api-e2e-derive-61e32580-76478fc64c-k86w9\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:03.995554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.995462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21f0d81e-708e-4f28-b554-c11c2e42c0ad-plugins-config-volume\") pod \"payload-processing-e2e-derive-61e32580-8599b8697-qk7tl\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") " pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:03.995554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:03.995508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfsb7\" (UniqueName: \"kubernetes.io/projected/21f0d81e-708e-4f28-b554-c11c2e42c0ad-kube-api-access-vfsb7\") pod \"payload-processing-e2e-derive-61e32580-8599b8697-qk7tl\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") " pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:04.096205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.096163    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21f0d81e-708e-4f28-b554-c11c2e42c0ad-plugins-config-volume\") pod \"payload-processing-e2e-derive-61e32580-8599b8697-qk7tl\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") " pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:04.096407 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.096227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfsb7\" (UniqueName: \"kubernetes.io/projected/21f0d81e-708e-4f28-b554-c11c2e42c0ad-kube-api-access-vfsb7\") pod \"payload-processing-e2e-derive-61e32580-8599b8697-qk7tl\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") " pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:04.096955 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.096920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21f0d81e-708e-4f28-b554-c11c2e42c0ad-plugins-config-volume\") pod \"payload-processing-e2e-derive-61e32580-8599b8697-qk7tl\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") " pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:04.098018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.098001    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:04.110347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.110329    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfsb7\" (UniqueName: \"kubernetes.io/projected/21f0d81e-708e-4f28-b554-c11c2e42c0ad-kube-api-access-vfsb7\") pod \"payload-processing-e2e-derive-61e32580-8599b8697-qk7tl\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") " pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:04.142828 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.142800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:04.150893 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.150637    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:04.385156 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.385134    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"]
Sep 19 09:28:04.387069 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:28:04.387040    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb WatchSource:0}: Error finding container 6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb: Status 404 returned error can't find the container with id 6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb
Sep 19 09:28:04.433607 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.433586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"]
Sep 19 09:28:04.435470 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:28:04.435444    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice/crio-f8ccc0fb79c4153a4aedf9747324637feeafe44a4ab081b72370db85af4cac62 WatchSource:0}: Error finding container f8ccc0fb79c4153a4aedf9747324637feeafe44a4ab081b72370db85af4cac62: Status 404 returned error can't find the container with id f8ccc0fb79c4153a4aedf9747324637feeafe44a4ab081b72370db85af4cac62
Sep 19 09:28:04.831954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.831867    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="152abdcd-512c-4de9-9b5f-9c68741bf6f0" path="/var/lib/kubelet/pods/152abdcd-512c-4de9-9b5f-9c68741bf6f0/volumes"
Sep 19 09:28:04.855566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:04.855528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:05.324737 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.324703    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl" event={"ID":"21f0d81e-708e-4f28-b554-c11c2e42c0ad","Type":"ContainerStarted","Data":"85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7"}
Sep 19 09:28:05.324979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.324765    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl" event={"ID":"21f0d81e-708e-4f28-b554-c11c2e42c0ad","Type":"ContainerStarted","Data":"f8ccc0fb79c4153a4aedf9747324637feeafe44a4ab081b72370db85af4cac62"}
Sep 19 09:28:05.324979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.324892    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:05.326394 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.326369    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9" event={"ID":"dc717f90-730f-4361-925d-510eae44dc5c","Type":"ContainerStarted","Data":"d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e"}
Sep 19 09:28:05.326525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.326416    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9" event={"ID":"dc717f90-730f-4361-925d-510eae44dc5c","Type":"ContainerStarted","Data":"6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb"}
Sep 19 09:28:05.326525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.326477    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:05.340670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.340608    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl" podStartSLOduration=2.34058853 podStartE2EDuration="2.34058853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:28:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:28:05.340049555 +0000 UTC m=+1299.026854853" watchObservedRunningTime="2026-09-19 09:28:05.34058853 +0000 UTC m=+1299.027393861"
Sep 19 09:28:05.360408 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.360355    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9" podStartSLOduration=1.884363687 podStartE2EDuration="2.360337278s" totalImagesPullingTime="475.973591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:28:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:28:05.358935396 +0000 UTC m=+1299.045740695" watchObservedRunningTime="2026-09-19 09:28:05.360337278 +0000 UTC m=+1299.047142576"
Sep 19 09:28:05.709959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:05.709920    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:06.405562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:06.405516    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:07.209002 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:07.208964    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:07.911357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:07.911316    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:08.606660 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:08.606625    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:09.368307 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:09.368276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:10.312857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:10.312814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:10.990790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:10.990747    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:11.343659 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:11.343576    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:11.698324 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:11.698284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:12.503338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:12.503299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:13.775286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:13.775250    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:14.615723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:14.615679    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:15.654155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:15.654086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:16.339822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:16.339794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:16.618165 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:16.618023    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:17.311383 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.311345    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:17.346766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.346731    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"]
Sep 19 09:28:17.347007 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.346981    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22" podUID="f8730d4c-92c0-4fad-becc-2a066444db6e" containerName="payload-pre-processing" containerID="cri-o://a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486" gracePeriod=30
Sep 19 09:28:17.413981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.413947    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:17.512393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.512359    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:17.609076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.609052    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:28:17.612032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.611999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:17.712583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.712537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:17.719537 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.719517    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klg28\" (UniqueName: \"kubernetes.io/projected/f8730d4c-92c0-4fad-becc-2a066444db6e-kube-api-access-klg28\") pod \"f8730d4c-92c0-4fad-becc-2a066444db6e\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") "
Sep 19 09:28:17.719652 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.719570    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8730d4c-92c0-4fad-becc-2a066444db6e-plugins-config-volume\") pod \"f8730d4c-92c0-4fad-becc-2a066444db6e\" (UID: \"f8730d4c-92c0-4fad-becc-2a066444db6e\") "
Sep 19 09:28:17.719969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.719935    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8730d4c-92c0-4fad-becc-2a066444db6e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8730d4c-92c0-4fad-becc-2a066444db6e" (UID: "f8730d4c-92c0-4fad-becc-2a066444db6e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:28:17.721881 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.721855    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8730d4c-92c0-4fad-becc-2a066444db6e-kube-api-access-klg28" (OuterVolumeSpecName: "kube-api-access-klg28") pod "f8730d4c-92c0-4fad-becc-2a066444db6e" (UID: "f8730d4c-92c0-4fad-becc-2a066444db6e"). InnerVolumeSpecName "kube-api-access-klg28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:28:17.820884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.820851    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8730d4c-92c0-4fad-becc-2a066444db6e-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:17.820884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.820882    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klg28\" (UniqueName: \"kubernetes.io/projected/f8730d4c-92c0-4fad-becc-2a066444db6e-kube-api-access-klg28\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:17.838481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:17.838401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:18.042915 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.042881    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:18.400837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.400803    2581 generic.go:356] "Generic (PLEG): container finished" podID="f8730d4c-92c0-4fad-becc-2a066444db6e" containerID="a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486" exitCode=0
Sep 19 09:28:18.401339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.400870    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"
Sep 19 09:28:18.401339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.400884    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22" event={"ID":"f8730d4c-92c0-4fad-becc-2a066444db6e","Type":"ContainerDied","Data":"a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486"}
Sep 19 09:28:18.401339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.400944    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22" event={"ID":"f8730d4c-92c0-4fad-becc-2a066444db6e","Type":"ContainerDied","Data":"c23fd44051ca0b4dd82c043678040844f8c67b515adabb9f811f3974952ef1ec"}
Sep 19 09:28:18.401339 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.400965    2581 scope.go:117] "RemoveContainer" containerID="a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486"
Sep 19 09:28:18.409961 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.409940    2581 scope.go:117] "RemoveContainer" containerID="a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486"
Sep 19 09:28:18.410407 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:18.410388    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486\": container with ID starting with a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486 not found: ID does not exist" containerID="a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486"
Sep 19 09:28:18.410491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.410427    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486"} err="failed to get container status \"a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486\": rpc error: code = NotFound desc = could not find container \"a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486\": container with ID starting with a5c3c37e217c355d1405389f38777327c7349c319fcd75d8aac9bdf282a1d486 not found: ID does not exist"
Sep 19 09:28:18.421954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.421930    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"]
Sep 19 09:28:18.425940 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.425908    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0ce808-564fbd897d-d4j22"]
Sep 19 09:28:18.830977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.830882    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8730d4c-92c0-4fad-becc-2a066444db6e" path="/var/lib/kubelet/pods/f8730d4c-92c0-4fad-becc-2a066444db6e/volumes"
Sep 19 09:28:18.922639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:18.922604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:19.889582 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:19.889550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:20.705639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:20.705604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:21.637944 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:21.637908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:22.414532 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:22.414502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:23.026807 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:23.026773    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"]
Sep 19 09:28:23.122307 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:23.122268    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:23.222628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:23.222588    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:24.226939 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:24.226901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:26.030244 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.030211    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"]
Sep 19 09:28:26.030668 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.030421    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9" podUID="dc717f90-730f-4361-925d-510eae44dc5c" containerName="maas-api" containerID="cri-o://d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e" gracePeriod=30
Sep 19 09:28:26.072367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.072339    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:26.107679 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.107634    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"]
Sep 19 09:28:26.107962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.107926    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl" podUID="21f0d81e-708e-4f28-b554-c11c2e42c0ad" containerName="payload-processing" containerID="cri-o://85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7" gracePeriod=30
Sep 19 09:28:26.173566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.173529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:26.273638 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.273604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:26.333744 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.333692    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:26.352921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.352896    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:26.379977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.379951    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:26.439292 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.439256    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:26.443390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.443365    2581 generic.go:356] "Generic (PLEG): container finished" podID="21f0d81e-708e-4f28-b554-c11c2e42c0ad" containerID="85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7" exitCode=0
Sep 19 09:28:26.443512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.443408    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"
Sep 19 09:28:26.443512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.443446    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl" event={"ID":"21f0d81e-708e-4f28-b554-c11c2e42c0ad","Type":"ContainerDied","Data":"85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7"}
Sep 19 09:28:26.443512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.443500    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl" event={"ID":"21f0d81e-708e-4f28-b554-c11c2e42c0ad","Type":"ContainerDied","Data":"f8ccc0fb79c4153a4aedf9747324637feeafe44a4ab081b72370db85af4cac62"}
Sep 19 09:28:26.443640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.443528    2581 scope.go:117] "RemoveContainer" containerID="85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7"
Sep 19 09:28:26.444805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.444788    2581 generic.go:356] "Generic (PLEG): container finished" podID="dc717f90-730f-4361-925d-510eae44dc5c" containerID="d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e" exitCode=0
Sep 19 09:28:26.444867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.444827    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"
Sep 19 09:28:26.444867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.444837    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9" event={"ID":"dc717f90-730f-4361-925d-510eae44dc5c","Type":"ContainerDied","Data":"d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e"}
Sep 19 09:28:26.444936 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.444866    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9" event={"ID":"dc717f90-730f-4361-925d-510eae44dc5c","Type":"ContainerDied","Data":"6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb"}
Sep 19 09:28:26.452447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.452431    2581 scope.go:117] "RemoveContainer" containerID="85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7"
Sep 19 09:28:26.452716 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:26.452695    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7\": container with ID starting with 85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7 not found: ID does not exist" containerID="85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7"
Sep 19 09:28:26.452764 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.452725    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7"} err="failed to get container status \"85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7\": rpc error: code = NotFound desc = could not find container \"85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7\": container with ID starting with 85358862e770ba1e081bed7cc72a3a5136e4ac060f2950d748150beca22341c7 not found: ID does not exist"
Sep 19 09:28:26.452764 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.452742    2581 scope.go:117] "RemoveContainer" containerID="d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e"
Sep 19 09:28:26.460859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.460842    2581 scope.go:117] "RemoveContainer" containerID="d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e"
Sep 19 09:28:26.461144 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:26.461127    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e\": container with ID starting with d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e not found: ID does not exist" containerID="d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e"
Sep 19 09:28:26.461202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.461151    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e"} err="failed to get container status \"d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e\": rpc error: code = NotFound desc = could not find container \"d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e\": container with ID starting with d1340a7a9b2eb76319a9bb704a4205f4471bebf7b9abd29d50f089afeea3f97e not found: ID does not exist"
Sep 19 09:28:26.506888 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.506850    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls\") pod \"dc717f90-730f-4361-925d-510eae44dc5c\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") "
Sep 19 09:28:26.507077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.506897    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc717f90-730f-4361-925d-510eae44dc5c-openshift-service-ca\") pod \"dc717f90-730f-4361-925d-510eae44dc5c\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") "
Sep 19 09:28:26.507077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.506939    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21f0d81e-708e-4f28-b554-c11c2e42c0ad-plugins-config-volume\") pod \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") "
Sep 19 09:28:26.507077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.507031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-metrics-certs\") pod \"dc717f90-730f-4361-925d-510eae44dc5c\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") "
Sep 19 09:28:26.507077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.507055    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfsb7\" (UniqueName: \"kubernetes.io/projected/21f0d81e-708e-4f28-b554-c11c2e42c0ad-kube-api-access-vfsb7\") pod \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\" (UID: \"21f0d81e-708e-4f28-b554-c11c2e42c0ad\") "
Sep 19 09:28:26.507331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.507136    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9llv\" (UniqueName: \"kubernetes.io/projected/dc717f90-730f-4361-925d-510eae44dc5c-kube-api-access-p9llv\") pod \"dc717f90-730f-4361-925d-510eae44dc5c\" (UID: \"dc717f90-730f-4361-925d-510eae44dc5c\") "
Sep 19 09:28:26.507415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.507386    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc717f90-730f-4361-925d-510eae44dc5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc717f90-730f-4361-925d-510eae44dc5c" (UID: "dc717f90-730f-4361-925d-510eae44dc5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:28:26.507465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.507448    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21f0d81e-708e-4f28-b554-c11c2e42c0ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "21f0d81e-708e-4f28-b554-c11c2e42c0ad" (UID: "21f0d81e-708e-4f28-b554-c11c2e42c0ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:28:26.509356 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.509330    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc717f90-730f-4361-925d-510eae44dc5c" (UID: "dc717f90-730f-4361-925d-510eae44dc5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:28:26.509356 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.509351    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f0d81e-708e-4f28-b554-c11c2e42c0ad-kube-api-access-vfsb7" (OuterVolumeSpecName: "kube-api-access-vfsb7") pod "21f0d81e-708e-4f28-b554-c11c2e42c0ad" (UID: "21f0d81e-708e-4f28-b554-c11c2e42c0ad"). InnerVolumeSpecName "kube-api-access-vfsb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:28:26.509505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.509350    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "dc717f90-730f-4361-925d-510eae44dc5c" (UID: "dc717f90-730f-4361-925d-510eae44dc5c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:28:26.509505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.509379    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc717f90-730f-4361-925d-510eae44dc5c-kube-api-access-p9llv" (OuterVolumeSpecName: "kube-api-access-p9llv") pod "dc717f90-730f-4361-925d-510eae44dc5c" (UID: "dc717f90-730f-4361-925d-510eae44dc5c"). InnerVolumeSpecName "kube-api-access-p9llv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:28:26.608223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.608089    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:26.608223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.608167    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfsb7\" (UniqueName: \"kubernetes.io/projected/21f0d81e-708e-4f28-b554-c11c2e42c0ad-kube-api-access-vfsb7\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:26.608223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.608178    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9llv\" (UniqueName: \"kubernetes.io/projected/dc717f90-730f-4361-925d-510eae44dc5c-kube-api-access-p9llv\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:26.608223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.608188    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc717f90-730f-4361-925d-510eae44dc5c-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:26.608223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.608196    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc717f90-730f-4361-925d-510eae44dc5c-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:26.608223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.608204    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21f0d81e-708e-4f28-b554-c11c2e42c0ad-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:26.791445 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.791410    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"]
Sep 19 09:28:26.800804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.800772    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9"]
Sep 19 09:28:26.823179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.823148    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"]
Sep 19 09:28:26.831538 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.831511    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc717f90-730f-4361-925d-510eae44dc5c" path="/var/lib/kubelet/pods/dc717f90-730f-4361-925d-510eae44dc5c/volumes"
Sep 19 09:28:26.844378 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.844354    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl"]
Sep 19 09:28:26.937531 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:26.937491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:27.144727 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:27.144632    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:27.236462 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:27.236417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:27.338587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:27.338540    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-derive-61e32580-8599b8697-qk7tl" podUID="21f0d81e-708e-4f28-b554-c11c2e42c0ad" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.52:9004: i/o timeout"
Sep 19 09:28:27.342817 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:27.342790    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:27.851463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:27.851423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:28.631962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:28.631923    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:28.742474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:28.742442    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:28.830991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:28.830955    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21f0d81e-708e-4f28-b554-c11c2e42c0ad" path="/var/lib/kubelet/pods/21f0d81e-708e-4f28-b554-c11c2e42c0ad/volumes"
Sep 19 09:28:29.338859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:29.338809    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-derive-61e32580-76478fc64c-k86w9" podUID="dc717f90-730f-4361-925d-510eae44dc5c" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.51:8443/health\": context deadline exceeded"
Sep 19 09:28:29.685158 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:29.685093    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:30.246086 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:30.246040    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache]"
Sep 19 09:28:30.605150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:30.605015    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:32.470583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:32.470545    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:28:33.489143 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:33.489082    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache]"
Sep 19 09:28:34.587332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:34.587291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:34.688348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:34.688318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:34.787517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:34.787483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:34.848463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:34.848353    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:35.259781 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:35.259744    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:38.046675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.046624    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb" podUID="60ec054a-f2e6-4f13-863f-a9cb183211bb" containerName="main" containerID="cri-o://88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6" gracePeriod=45
Sep 19 09:28:38.297166 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.297090    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:28:38.307275 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307252    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-tmp-dir\") pod \"60ec054a-f2e6-4f13-863f-a9cb183211bb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") "
Sep 19 09:28:38.307406 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307296    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60ec054a-f2e6-4f13-863f-a9cb183211bb-tls-certs\") pod \"60ec054a-f2e6-4f13-863f-a9cb183211bb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") "
Sep 19 09:28:38.307406 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307349    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-dshm\") pod \"60ec054a-f2e6-4f13-863f-a9cb183211bb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") "
Sep 19 09:28:38.307406 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307374    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-home\") pod \"60ec054a-f2e6-4f13-863f-a9cb183211bb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") "
Sep 19 09:28:38.307573 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307412    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-model-cache\") pod \"60ec054a-f2e6-4f13-863f-a9cb183211bb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") "
Sep 19 09:28:38.307573 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307457    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2kw8\" (UniqueName: \"kubernetes.io/projected/60ec054a-f2e6-4f13-863f-a9cb183211bb-kube-api-access-x2kw8\") pod \"60ec054a-f2e6-4f13-863f-a9cb183211bb\" (UID: \"60ec054a-f2e6-4f13-863f-a9cb183211bb\") "
Sep 19 09:28:38.307671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307564    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60ec054a-f2e6-4f13-863f-a9cb183211bb" (UID: "60ec054a-f2e6-4f13-863f-a9cb183211bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:38.307723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307668    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-home" (OuterVolumeSpecName: "home") pod "60ec054a-f2e6-4f13-863f-a9cb183211bb" (UID: "60ec054a-f2e6-4f13-863f-a9cb183211bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:38.307723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307705    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "60ec054a-f2e6-4f13-863f-a9cb183211bb" (UID: "60ec054a-f2e6-4f13-863f-a9cb183211bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:38.307813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.307759    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:38.309880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.309851    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ec054a-f2e6-4f13-863f-a9cb183211bb-kube-api-access-x2kw8" (OuterVolumeSpecName: "kube-api-access-x2kw8") pod "60ec054a-f2e6-4f13-863f-a9cb183211bb" (UID: "60ec054a-f2e6-4f13-863f-a9cb183211bb"). InnerVolumeSpecName "kube-api-access-x2kw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:28:38.310085 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.310058    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-dshm" (OuterVolumeSpecName: "dshm") pod "60ec054a-f2e6-4f13-863f-a9cb183211bb" (UID: "60ec054a-f2e6-4f13-863f-a9cb183211bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:28:38.310085 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.310072    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60ec054a-f2e6-4f13-863f-a9cb183211bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60ec054a-f2e6-4f13-863f-a9cb183211bb" (UID: "60ec054a-f2e6-4f13-863f-a9cb183211bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:28:38.408930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.408893    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:38.408930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.408921    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:38.408930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.408930    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60ec054a-f2e6-4f13-863f-a9cb183211bb-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:38.408930 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.408940    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2kw8\" (UniqueName: \"kubernetes.io/projected/60ec054a-f2e6-4f13-863f-a9cb183211bb-kube-api-access-x2kw8\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:38.409226 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.408949    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60ec054a-f2e6-4f13-863f-a9cb183211bb-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:28:38.504734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.504704    2581 generic.go:356] "Generic (PLEG): container finished" podID="60ec054a-f2e6-4f13-863f-a9cb183211bb" containerID="88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6" exitCode=0
Sep 19 09:28:38.504905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.504749    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb" event={"ID":"60ec054a-f2e6-4f13-863f-a9cb183211bb","Type":"ContainerDied","Data":"88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6"}
Sep 19 09:28:38.504905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.504782    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb" event={"ID":"60ec054a-f2e6-4f13-863f-a9cb183211bb","Type":"ContainerDied","Data":"8006813ac3e5516766df6e5b965f74a08ceb3c75c8440a91eeb2e7fa7fda0516"}
Sep 19 09:28:38.504905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.504800    2581 scope.go:117] "RemoveContainer" containerID="88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6"
Sep 19 09:28:38.504905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.504749    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"
Sep 19 09:28:38.514401 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.514381    2581 scope.go:117] "RemoveContainer" containerID="88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6"
Sep 19 09:28:38.514712 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:38.514692    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6\": container with ID starting with 88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6 not found: ID does not exist" containerID="88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6"
Sep 19 09:28:38.514777 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.514725    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6"} err="failed to get container status \"88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6\": rpc error: code = NotFound desc = could not find container \"88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6\": container with ID starting with 88844e416cf67ccbcf363c05b7bd4e4957a00111b663a469a3992d616dff7cc6 not found: ID does not exist"
Sep 19 09:28:38.525114 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.525064    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"]
Sep 19 09:28:38.528739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.528705    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-0ce808/autores-model-0ce808-kserve-6f5fcf77c9-f5sdb"]
Sep 19 09:28:38.831564 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:38.831524    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60ec054a-f2e6-4f13-863f-a9cb183211bb" path="/var/lib/kubelet/pods/60ec054a-f2e6-4f13-863f-a9cb183211bb/volumes"
Sep 19 09:28:40.287339 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:40.287297    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 09:28:48.281689 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:48.281601    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:28:48.512980 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:48.512940    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 09:28:50.296895 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:28:50.296855    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 09:28:59.529428 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:28:59.529379    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:00.153945 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:00.153908    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:00.341581 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:00.341542    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 09:29:01.181675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:01.181638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:03.491368 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:03.491323    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 09:29:03.949432 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:03.949382    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache]"
Sep 19 09:29:03.950292 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:03.950258    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache]"
Sep 19 09:29:07.034815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:07.034780    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:07.104312 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:07.104271    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:07.223642 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:07.223604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:07.330447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:07.330351    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:10.382679 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:10.382640    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache]"
Sep 19 09:29:18.524490 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:18.524449    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache]"
Sep 19 09:29:20.391818 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:20.391780    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f0d81e_708e_4f28_b554_c11c2e42c0ad.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc717f90_730f_4361_925d_510eae44dc5c.slice/crio-6bae54f7d0ae87f762dd782aedd389e2ec44e8bab21793bf8204c89a708beabb\": RecentStats: unable to find data in memory cache]"
Sep 19 09:29:20.877278 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.877236    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:20.994021 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.993990    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"]
Sep 19 09:29:20.994326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994314    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc717f90-730f-4361-925d-510eae44dc5c" containerName="maas-api"
Sep 19 09:29:20.994373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994328    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc717f90-730f-4361-925d-510eae44dc5c" containerName="maas-api"
Sep 19 09:29:20.994373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994340    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8730d4c-92c0-4fad-becc-2a066444db6e" containerName="payload-pre-processing"
Sep 19 09:29:20.994373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994346    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8730d4c-92c0-4fad-becc-2a066444db6e" containerName="payload-pre-processing"
Sep 19 09:29:20.994373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994352    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f0d81e-708e-4f28-b554-c11c2e42c0ad" containerName="payload-processing"
Sep 19 09:29:20.994373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994357    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f0d81e-708e-4f28-b554-c11c2e42c0ad" containerName="payload-processing"
Sep 19 09:29:20.994373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994369    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60ec054a-f2e6-4f13-863f-a9cb183211bb" containerName="main"
Sep 19 09:29:20.994373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994375    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="60ec054a-f2e6-4f13-863f-a9cb183211bb" containerName="main"
Sep 19 09:29:20.994630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994438    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc717f90-730f-4361-925d-510eae44dc5c" containerName="maas-api"
Sep 19 09:29:20.994630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994449    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8730d4c-92c0-4fad-becc-2a066444db6e" containerName="payload-pre-processing"
Sep 19 09:29:20.994630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994456    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f0d81e-708e-4f28-b554-c11c2e42c0ad" containerName="payload-processing"
Sep 19 09:29:20.994630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.994460    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="60ec054a-f2e6-4f13-863f-a9cb183211bb" containerName="main"
Sep 19 09:29:20.997293 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.997276    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"
Sep 19 09:29:20.999899 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:20.999874    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:29:21.000089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.000076    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v7qlc\""
Sep 19 09:29:21.007887 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.007857    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"]
Sep 19 09:29:21.082877 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.082838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fe6e0-59b1-49ae-a425-0081cafb90d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8\" (UID: \"8d3fe6e0-59b1-49ae-a425-0081cafb90d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"
Sep 19 09:29:21.183584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.183487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fe6e0-59b1-49ae-a425-0081cafb90d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8\" (UID: \"8d3fe6e0-59b1-49ae-a425-0081cafb90d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"
Sep 19 09:29:21.184192 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.184171    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fe6e0-59b1-49ae-a425-0081cafb90d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8\" (UID: \"8d3fe6e0-59b1-49ae-a425-0081cafb90d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"
Sep 19 09:29:21.202487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.202434    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:21.309083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.309048    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"
Sep 19 09:29:21.583077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.583049    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"]
Sep 19 09:29:21.585255 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:29:21.585226    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3fe6e0_59b1_49ae_a425_0081cafb90d8.slice/crio-8ece7785160ae22962393eaba348220d08de2984249284ce8d75e33a93749673 WatchSource:0}: Error finding container 8ece7785160ae22962393eaba348220d08de2984249284ce8d75e33a93749673: Status 404 returned error can't find the container with id 8ece7785160ae22962393eaba348220d08de2984249284ce8d75e33a93749673
Sep 19 09:29:21.732610 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.732573    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8" event={"ID":"8d3fe6e0-59b1-49ae-a425-0081cafb90d8","Type":"ContainerStarted","Data":"8ece7785160ae22962393eaba348220d08de2984249284ce8d75e33a93749673"}
Sep 19 09:29:21.944358 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:21.944324    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:22.708469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:22.708421    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:22.738538 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:22.738499    2581 generic.go:356] "Generic (PLEG): container finished" podID="8d3fe6e0-59b1-49ae-a425-0081cafb90d8" containerID="4626bf6dd6073ab019d02ef0b3d3dfe72e7cf34c500f39bcfc0a8fef6b75459f" exitCode=0
Sep 19 09:29:22.738706 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:22.738553    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8" event={"ID":"8d3fe6e0-59b1-49ae-a425-0081cafb90d8","Type":"ContainerDied","Data":"4626bf6dd6073ab019d02ef0b3d3dfe72e7cf34c500f39bcfc0a8fef6b75459f"}
Sep 19 09:29:23.081939 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:23.081842    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:23.369854 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:23.369757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:23.468021 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:23.467986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:23.635197 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:23.635159    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:23.875376 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:23.875352    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"
Sep 19 09:29:24.011907 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.011810    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fe6e0-59b1-49ae-a425-0081cafb90d8-maas-api-service-ca\") pod \"8d3fe6e0-59b1-49ae-a425-0081cafb90d8\" (UID: \"8d3fe6e0-59b1-49ae-a425-0081cafb90d8\") "
Sep 19 09:29:24.012217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.012191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d3fe6e0-59b1-49ae-a425-0081cafb90d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d3fe6e0-59b1-49ae-a425-0081cafb90d8" (UID: "8d3fe6e0-59b1-49ae-a425-0081cafb90d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:29:24.065705 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.065652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:24.113182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.113135    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d3fe6e0-59b1-49ae-a425-0081cafb90d8-maas-api-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:24.277272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.277189    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:24.360835 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.360803    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"]
Sep 19 09:29:24.361169 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.361154    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d3fe6e0-59b1-49ae-a425-0081cafb90d8" containerName="revoke-keys"
Sep 19 09:29:24.361224 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.361170    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d3fe6e0-59b1-49ae-a425-0081cafb90d8" containerName="revoke-keys"
Sep 19 09:29:24.361267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.361230    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d3fe6e0-59b1-49ae-a425-0081cafb90d8" containerName="revoke-keys"
Sep 19 09:29:24.364160 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.364143    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.369693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.369671    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 09:29:24.369819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.369682    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-ac387c-openshift-default-dockercfg-f7wvr\""
Sep 19 09:29:24.382317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.382292    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"]
Sep 19 09:29:24.403926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.403890    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:24.516474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516435    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-credential-socket\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516472    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-data\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516493    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istiod-ca-cert\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516511    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-socket\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516595    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-token\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516628    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-podinfo\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-envoy\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516737    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xds7\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-kube-api-access-4xds7\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.516821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.516766    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-certs\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-socket\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617348    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-token\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617371    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-podinfo\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-envoy\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xds7\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-kube-api-access-4xds7\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-certs\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617502    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-credential-socket\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-data\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617551    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istiod-ca-cert\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.617979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.617779    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-socket\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.618473 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.618449    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-certs\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.619417 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.619391    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-data\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.619550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.619464    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-credential-socket\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.619716 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.619694    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istiod-ca-cert\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.619906 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.619891    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-envoy\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.620162 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.620146    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-podinfo\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.625988 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.625963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-token\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.626314 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.626296    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xds7\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-kube-api-access-4xds7\") pod \"e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") " pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.674225 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.674197    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:24.750938 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.750909    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"
Sep 19 09:29:24.751114 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.750936    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8" event={"ID":"8d3fe6e0-59b1-49ae-a425-0081cafb90d8","Type":"ContainerDied","Data":"8ece7785160ae22962393eaba348220d08de2984249284ce8d75e33a93749673"}
Sep 19 09:29:24.751114 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.750985    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ece7785160ae22962393eaba348220d08de2984249284ce8d75e33a93749673"
Sep 19 09:29:24.783598 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.783557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:24.952547 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.952485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"]
Sep 19 09:29:24.955067 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:29:24.955043    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2b09d7b_2b5d_4978_89dc_7a7050fbfeb7.slice/crio-58e3d1f34aa5fde8d2ee5c0b53574dc122db5379f5370245a25c213093557d23 WatchSource:0}: Error finding container 58e3d1f34aa5fde8d2ee5c0b53574dc122db5379f5370245a25c213093557d23: Status 404 returned error can't find the container with id 58e3d1f34aa5fde8d2ee5c0b53574dc122db5379f5370245a25c213093557d23
Sep 19 09:29:24.957954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.957915    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:29:24.958042 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.958003    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:29:24.958090 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:24.958048    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:29:25.098395 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:25.098343    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:25.763367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:25.763330    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" event={"ID":"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7","Type":"ContainerStarted","Data":"a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947"}
Sep 19 09:29:25.763552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:25.763420    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" event={"ID":"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7","Type":"ContainerStarted","Data":"58e3d1f34aa5fde8d2ee5c0b53574dc122db5379f5370245a25c213093557d23"}
Sep 19 09:29:25.784865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:25.784813    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" podStartSLOduration=1.784797031 podStartE2EDuration="1.784797031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:29:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:29:25.784077138 +0000 UTC m=+1379.470882437" watchObservedRunningTime="2026-09-19 09:29:25.784797031 +0000 UTC m=+1379.471602389"
Sep 19 09:29:25.858034 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:25.857998    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"]
Sep 19 09:29:25.904582 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:25.904546    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:26.675287 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:26.675247    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:26.681357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:26.681333    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:26.768423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:26.768398    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:26.769325 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:26.769303    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:27.123344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:27.123310    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:27.749755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:27.749719    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:27.836588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:27.836544    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:27.937665 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:27.937635    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:28.641575 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:28.641541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:29.534622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:29.534586    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:30.037239 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:30.037209    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:30.140835 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:30.140800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:30.238651 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:30.238619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:30.491287 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:30.491248    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:31.244720 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.244682    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:31.316585 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.316535    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:31.425812 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.425770    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:31.800376 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.800323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:31.901293 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.901264    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:31.985864 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.985825    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"]
Sep 19 09:29:31.990497 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.990480    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:31.993020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.992995    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-ac387c-dockercfg-d94q5\""
Sep 19 09:29:31.993020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:31.993014    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-ac387c\""
Sep 19 09:29:32.000405 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.000383    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"]
Sep 19 09:29:32.080579 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.080484    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjcvh\" (UniqueName: \"kubernetes.io/projected/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-kube-api-access-rjcvh\") pod \"payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.080579 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.080552    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.181910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.181871    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.182145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.181973    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjcvh\" (UniqueName: \"kubernetes.io/projected/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-kube-api-access-rjcvh\") pod \"payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.182622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.182602    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.222873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.222843    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjcvh\" (UniqueName: \"kubernetes.io/projected/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-kube-api-access-rjcvh\") pod \"payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.247005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.246971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:32.301872 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.301841    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.350211 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.350130    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"]
Sep 19 09:29:32.350381 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.350353    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj" podUID="b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" containerName="payload-processing" containerID="cri-o://f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30" gracePeriod=30
Sep 19 09:29:32.418571 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.418531    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:32.497384 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.497340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:32.588040 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.587947    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"]
Sep 19 09:29:32.590559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.590537    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"]
Sep 19 09:29:32.591905 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:29:32.591875    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ea1ec1_e542_4f94_a9e3_89a2ff7a249b.slice/crio-8ff95731dbd9f873f446d69c1761654fe7e1efe649e119187d2933baf42bcaa5 WatchSource:0}: Error finding container 8ff95731dbd9f873f446d69c1761654fe7e1efe649e119187d2933baf42bcaa5: Status 404 returned error can't find the container with id 8ff95731dbd9f873f446d69c1761654fe7e1efe649e119187d2933baf42bcaa5
Sep 19 09:29:32.594373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.594351    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-acc640-g8mz8"]
Sep 19 09:29:32.595210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.595186    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:29:32.595462 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.595443    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:32.669038 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.668998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:32.687274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.687243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6vx6\" (UniqueName: \"kubernetes.io/projected/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-kube-api-access-t6vx6\") pod \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") "
Sep 19 09:29:32.687436 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.687324    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-plugins-config-volume\") pod \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\" (UID: \"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad\") "
Sep 19 09:29:32.687729 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.687698    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" (UID: "b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:29:32.689683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.689655    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-kube-api-access-t6vx6" (OuterVolumeSpecName: "kube-api-access-t6vx6") pod "b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" (UID: "b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad"). InnerVolumeSpecName "kube-api-access-t6vx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:29:32.788405 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.788370    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:32.788405 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.788398    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6vx6\" (UniqueName: \"kubernetes.io/projected/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad-kube-api-access-t6vx6\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:32.802298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.802269    2581 generic.go:356] "Generic (PLEG): container finished" podID="b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" containerID="f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30" exitCode=0
Sep 19 09:29:32.802487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.802333    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"
Sep 19 09:29:32.802487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.802353    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj" event={"ID":"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad","Type":"ContainerDied","Data":"f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30"}
Sep 19 09:29:32.802487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.802404    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj" event={"ID":"b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad","Type":"ContainerDied","Data":"b6a2a96be725b94bfd20e8f7690801bcc7748c56af56283d23b46e5e2e675dc2"}
Sep 19 09:29:32.802487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.802427    2581 scope.go:117] "RemoveContainer" containerID="f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30"
Sep 19 09:29:32.804232 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.804209    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk" event={"ID":"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b","Type":"ContainerStarted","Data":"0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e"}
Sep 19 09:29:32.804350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.804254    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk" event={"ID":"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b","Type":"ContainerStarted","Data":"8ff95731dbd9f873f446d69c1761654fe7e1efe649e119187d2933baf42bcaa5"}
Sep 19 09:29:32.804350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.804306    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:32.811391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.811370    2581 scope.go:117] "RemoveContainer" containerID="f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30"
Sep 19 09:29:32.811691 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:32.811668    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30\": container with ID starting with f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30 not found: ID does not exist" containerID="f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30"
Sep 19 09:29:32.811772 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.811702    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30"} err="failed to get container status \"f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30\": rpc error: code = NotFound desc = could not find container \"f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30\": container with ID starting with f81f92aa2dfce63f1657c421e0483faad09714444ab27ddddb9c2c554809ec30 not found: ID does not exist"
Sep 19 09:29:32.831267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.831220    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk" podStartSLOduration=1.831205196 podStartE2EDuration="1.831205196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:29:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:29:32.828780323 +0000 UTC m=+1386.515585623" watchObservedRunningTime="2026-09-19 09:29:32.831205196 +0000 UTC m=+1386.518010493"
Sep 19 09:29:32.832813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.832792    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d3fe6e0-59b1-49ae-a425-0081cafb90d8" path="/var/lib/kubelet/pods/8d3fe6e0-59b1-49ae-a425-0081cafb90d8/volumes"
Sep 19 09:29:32.842826 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.842800    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"]
Sep 19 09:29:32.848991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.848966    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-acc640-6d6978ccb5-5gspj"]
Sep 19 09:29:32.941900 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:32.941864    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:33.364248 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:33.364211    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:34.182709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:34.182673    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:34.831294 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:34.831262    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" path="/var/lib/kubelet/pods/b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad/volumes"
Sep 19 09:29:34.871503 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:34.871465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:35.572164 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:35.572131    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:36.419495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:36.419456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:38.757416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:38.757376    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:38.857479 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:38.857438    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:39.150393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:39.150355    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:40.716389 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:40.716353    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:40.877805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:40.877767    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8" podUID="c7696837-5859-4494-9d90-2e0099b03c7d" containerName="main" containerID="cri-o://ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e" gracePeriod=45
Sep 19 09:29:41.123670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.123644    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:29:41.165457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165418    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-home\") pod \"c7696837-5859-4494-9d90-2e0099b03c7d\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") "
Sep 19 09:29:41.165626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165515    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29mww\" (UniqueName: \"kubernetes.io/projected/c7696837-5859-4494-9d90-2e0099b03c7d-kube-api-access-29mww\") pod \"c7696837-5859-4494-9d90-2e0099b03c7d\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") "
Sep 19 09:29:41.165626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165553    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7696837-5859-4494-9d90-2e0099b03c7d-tls-certs\") pod \"c7696837-5859-4494-9d90-2e0099b03c7d\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") "
Sep 19 09:29:41.165626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165584    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-model-cache\") pod \"c7696837-5859-4494-9d90-2e0099b03c7d\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") "
Sep 19 09:29:41.165754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165623    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-tmp-dir\") pod \"c7696837-5859-4494-9d90-2e0099b03c7d\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") "
Sep 19 09:29:41.165754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-dshm\") pod \"c7696837-5859-4494-9d90-2e0099b03c7d\" (UID: \"c7696837-5859-4494-9d90-2e0099b03c7d\") "
Sep 19 09:29:41.165754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165740    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-home" (OuterVolumeSpecName: "home") pod "c7696837-5859-4494-9d90-2e0099b03c7d" (UID: "c7696837-5859-4494-9d90-2e0099b03c7d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:29:41.165915 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165888    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7696837-5859-4494-9d90-2e0099b03c7d" (UID: "c7696837-5859-4494-9d90-2e0099b03c7d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:29:41.165994 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165968    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c7696837-5859-4494-9d90-2e0099b03c7d" (UID: "c7696837-5859-4494-9d90-2e0099b03c7d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:29:41.166047 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.165996    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:41.166047 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.166012    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:41.168142 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.168090    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7696837-5859-4494-9d90-2e0099b03c7d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7696837-5859-4494-9d90-2e0099b03c7d" (UID: "c7696837-5859-4494-9d90-2e0099b03c7d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:29:41.168257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.168152    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-dshm" (OuterVolumeSpecName: "dshm") pod "c7696837-5859-4494-9d90-2e0099b03c7d" (UID: "c7696837-5859-4494-9d90-2e0099b03c7d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:29:41.168257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.168190    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7696837-5859-4494-9d90-2e0099b03c7d-kube-api-access-29mww" (OuterVolumeSpecName: "kube-api-access-29mww") pod "c7696837-5859-4494-9d90-2e0099b03c7d" (UID: "c7696837-5859-4494-9d90-2e0099b03c7d"). InnerVolumeSpecName "kube-api-access-29mww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:29:41.266828 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.266731    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:41.266828 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.266767    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7696837-5859-4494-9d90-2e0099b03c7d-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:41.266828 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.266777    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29mww\" (UniqueName: \"kubernetes.io/projected/c7696837-5859-4494-9d90-2e0099b03c7d-kube-api-access-29mww\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:41.266828 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.266786    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7696837-5859-4494-9d90-2e0099b03c7d-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:29:41.520072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.519981    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:41.857939 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.857849    2581 generic.go:356] "Generic (PLEG): container finished" podID="c7696837-5859-4494-9d90-2e0099b03c7d" containerID="ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e" exitCode=0
Sep 19 09:29:41.857939 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.857893    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"
Sep 19 09:29:41.857939 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.857933    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8" event={"ID":"c7696837-5859-4494-9d90-2e0099b03c7d","Type":"ContainerDied","Data":"ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e"}
Sep 19 09:29:41.858667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.857976    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8" event={"ID":"c7696837-5859-4494-9d90-2e0099b03c7d","Type":"ContainerDied","Data":"de4675a49cf316bd693aef4ee1a311452eab144d7d67a87d733dfc1228965cd8"}
Sep 19 09:29:41.858667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.857998    2581 scope.go:117] "RemoveContainer" containerID="ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e"
Sep 19 09:29:41.867340 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.867328    2581 scope.go:117] "RemoveContainer" containerID="ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e"
Sep 19 09:29:41.867645 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:29:41.867627    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e\": container with ID starting with ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e not found: ID does not exist" containerID="ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e"
Sep 19 09:29:41.867734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.867652    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e"} err="failed to get container status \"ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e\": rpc error: code = NotFound desc = could not find container \"ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e\": container with ID starting with ea31788ed7b99e25504451f536cc009182801c0686886e5b9a2d62c0c716463e not found: ID does not exist"
Sep 19 09:29:41.880946 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.880914    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"]
Sep 19 09:29:41.883588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:41.883564    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-acc640/simulator-w4-acc640-kserve-84f6bb7c4b-z7ht8"]
Sep 19 09:29:42.830872 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:42.830838    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7696837-5859-4494-9d90-2e0099b03c7d" path="/var/lib/kubelet/pods/c7696837-5859-4494-9d90-2e0099b03c7d/volumes"
Sep 19 09:29:43.078633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:43.078597    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:29:43.813665 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:43.813637    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:29:47.304666 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:47.304600    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:47.391215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:47.391170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:47.487659 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:47.487617    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:47.589010 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:47.588909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:48.327465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:48.327424    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:48.484928 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:48.484895    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:49.872349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:49.872304    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:53.379937 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:53.379909    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:53.934078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:53.934049    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerID="a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947" exitCode=0
Sep 19 09:29:53.934280 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:53.934134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" event={"ID":"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7","Type":"ContainerDied","Data":"a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947"}
Sep 19 09:29:53.934554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:53.934535    2581 scope.go:117] "RemoveContainer" containerID="a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947"
Sep 19 09:29:53.935731 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:53.935689    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:29:53.935795 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:53.935783    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:29:53.935832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:53.935815    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:29:54.215823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:54.215740    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:54.941512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:54.941470    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" event={"ID":"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7","Type":"ContainerStarted","Data":"2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62"}
Sep 19 09:29:54.942628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:54.942599    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:55.668185 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:55.668141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:55.674684 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:55.674654    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:55.675867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:55.675842    2581 patch_prober.go:28] interesting pod/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 19 09:29:55.675963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:55.675907    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 19 09:29:56.523222 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.523185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:56.675435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.675393    2581 patch_prober.go:28] interesting pod/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 19 09:29:56.675607 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.675451    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 19 09:29:56.707125 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.707066    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:56.769286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.769249    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:29:56.804959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.804858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:56.961837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.961805    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"]
Sep 19 09:29:56.962164 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.962152    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7696837-5859-4494-9d90-2e0099b03c7d" containerName="main"
Sep 19 09:29:56.962208 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.962166    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7696837-5859-4494-9d90-2e0099b03c7d" containerName="main"
Sep 19 09:29:56.962208 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.962180    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" containerName="payload-processing"
Sep 19 09:29:56.962208 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.962185    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" containerName="payload-processing"
Sep 19 09:29:56.962311 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.962239    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7696837-5859-4494-9d90-2e0099b03c7d" containerName="main"
Sep 19 09:29:56.962311 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.962246    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9b9c40b-89ee-452e-98d4-0a5f7a6d67ad" containerName="payload-processing"
Sep 19 09:29:56.965059 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.965043    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:56.967846 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.967830    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"premium-w0-14e3f6-kserve-self-signed-certs\""
Sep 19 09:29:56.975702 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:56.975676    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"]
Sep 19 09:29:57.065397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.065299    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:57.124372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.124333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-dshm\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.124562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.124382    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg7zk\" (UniqueName: \"kubernetes.io/projected/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-kube-api-access-dg7zk\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.124562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.124436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-model-cache\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.124562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.124480    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-home\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.124562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.124501    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tmp-dir\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.124562 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.124523    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tls-certs\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.225492 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.225453    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg7zk\" (UniqueName: \"kubernetes.io/projected/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-kube-api-access-dg7zk\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.225492 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.225500    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-model-cache\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.225738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.225525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-home\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.225738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.225647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tmp-dir\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.225738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.225689    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tls-certs\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.225869 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.225797    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-dshm\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.225954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.225934    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-model-cache\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.226035 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.226007    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tmp-dir\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.226128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.226034    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-home\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.228294 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.228271    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-dshm\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.228465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.228446    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tls-certs\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.234066 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.234035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg7zk\" (UniqueName: \"kubernetes.io/projected/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-kube-api-access-dg7zk\") pod \"premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") " pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.275138 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.275083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:29:57.359514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.359486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:57.575379 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.575352    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"]
Sep 19 09:29:57.577436 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:29:57.577405    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63cf5320_3ebf_4c2a_a3ce_b27e277d407c.slice/crio-048be7b3a0c34ecb0318883075a21f5c718a2b4410aa70e2226507fb79480f74 WatchSource:0}: Error finding container 048be7b3a0c34ecb0318883075a21f5c718a2b4410aa70e2226507fb79480f74: Status 404 returned error can't find the container with id 048be7b3a0c34ecb0318883075a21f5c718a2b4410aa70e2226507fb79480f74
Sep 19 09:29:57.675289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.675249    2581 patch_prober.go:28] interesting pod/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 19 09:29:57.675474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.675322    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 19 09:29:57.749839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.749801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:29:57.960313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.960281    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg" event={"ID":"63cf5320-3ebf-4c2a-a3ce-b27e277d407c","Type":"ContainerStarted","Data":"204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712"}
Sep 19 09:29:57.960313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.960318    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg" event={"ID":"63cf5320-3ebf-4c2a-a3ce-b27e277d407c","Type":"ContainerStarted","Data":"048be7b3a0c34ecb0318883075a21f5c718a2b4410aa70e2226507fb79480f74"}
Sep 19 09:29:57.978501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:57.978446    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg" podStartSLOduration=1.9784302459999998 podStartE2EDuration="1.978430246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:29:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:29:57.976836941 +0000 UTC m=+1411.663642248" watchObservedRunningTime="2026-09-19 09:29:57.978430246 +0000 UTC m=+1411.665235543"
Sep 19 09:29:58.674774 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:58.674735    2581 patch_prober.go:28] interesting pod/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 19 09:29:58.675183 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:58.674790    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 19 09:29:59.674710 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:59.674675    2581 patch_prober.go:28] interesting pod/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 19 09:29:59.674933 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:29:59.674733    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 19 09:30:00.166309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.166272    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"]
Sep 19 09:30:00.169643 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.169620    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"
Sep 19 09:30:00.172540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.172516    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v7qlc\""
Sep 19 09:30:00.180080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.180054    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"]
Sep 19 09:30:00.180895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.180874    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"
Sep 19 09:30:00.184119 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.183978    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"]
Sep 19 09:30:00.184195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.184164    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"
Sep 19 09:30:00.187795 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.187771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"]
Sep 19 09:30:00.187934 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.187868    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"
Sep 19 09:30:00.203126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.203081    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"]
Sep 19 09:30:00.206071 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.206040    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"]
Sep 19 09:30:00.215165 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.215142    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"
Sep 19 09:30:00.222695 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.222660    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"
Sep 19 09:30:00.235605 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.235572    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:00.468301 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.468277    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"]
Sep 19 09:30:00.470496 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:30:00.470473    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb58fcb_45fc_44c6_bd06_690de28d1767.slice/crio-d0b45af07cd27d75389acc9618d76b138b0672940e3913280d26458b68fd8888 WatchSource:0}: Error finding container d0b45af07cd27d75389acc9618d76b138b0672940e3913280d26458b68fd8888: Status 404 returned error can't find the container with id d0b45af07cd27d75389acc9618d76b138b0672940e3913280d26458b68fd8888
Sep 19 09:30:00.512000 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.511951    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"]
Sep 19 09:30:00.515197 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:30:00.515172    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d66c1c_7ce4_4563_91cd_8226ac101f1e.slice/crio-f3ddafec6b0a4201ec5b4063dba2ddb594ba8cdc99adcb00effc42b66af8a361 WatchSource:0}: Error finding container f3ddafec6b0a4201ec5b4063dba2ddb594ba8cdc99adcb00effc42b66af8a361: Status 404 returned error can't find the container with id f3ddafec6b0a4201ec5b4063dba2ddb594ba8cdc99adcb00effc42b66af8a361
Sep 19 09:30:00.516480 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.516383    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"]
Sep 19 09:30:00.518658 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:30:00.518636    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bc30f5_d5dd_493f_a2ae_f71a0e1b0301.slice/crio-a708e1ae96fb18e1937e7694cb36a894c11e7ddc2ded760dbfb1e5ffa1ed0e83 WatchSource:0}: Error finding container a708e1ae96fb18e1937e7694cb36a894c11e7ddc2ded760dbfb1e5ffa1ed0e83: Status 404 returned error can't find the container with id a708e1ae96fb18e1937e7694cb36a894c11e7ddc2ded760dbfb1e5ffa1ed0e83
Sep 19 09:30:00.679574 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.679545    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:30:00.680586 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.680567    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:30:00.977700 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.977661    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r" event={"ID":"d4d66c1c-7ce4-4563-91cd-8226ac101f1e","Type":"ContainerStarted","Data":"eab5c15694c7a02ccb30bf7c3a6d7f45e2a54f80ea7a9dba236dbb20cf4014be"}
Sep 19 09:30:00.977898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.977721    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r" event={"ID":"d4d66c1c-7ce4-4563-91cd-8226ac101f1e","Type":"ContainerStarted","Data":"f3ddafec6b0a4201ec5b4063dba2ddb594ba8cdc99adcb00effc42b66af8a361"}
Sep 19 09:30:00.979369 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.979341    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt" event={"ID":"f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301","Type":"ContainerStarted","Data":"37f93a2f0655fa9324a57576b58146f1fbc6fcbeb9f1032b37e12d45adb8e5a0"}
Sep 19 09:30:00.979501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.979395    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt" event={"ID":"f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301","Type":"ContainerStarted","Data":"a708e1ae96fb18e1937e7694cb36a894c11e7ddc2ded760dbfb1e5ffa1ed0e83"}
Sep 19 09:30:00.980768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.980739    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq" event={"ID":"7bb58fcb-45fc-44c6-bd06-690de28d1767","Type":"ContainerStarted","Data":"dbebf3a7e5c43edf3c4873f301fb26fb88e7b619a8bdec83bb7a024e70951c19"}
Sep 19 09:30:00.980852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:00.980787    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq" event={"ID":"7bb58fcb-45fc-44c6-bd06-690de28d1767","Type":"ContainerStarted","Data":"d0b45af07cd27d75389acc9618d76b138b0672940e3913280d26458b68fd8888"}
Sep 19 09:30:01.022744 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.022694    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq" podStartSLOduration=1.02267745 podStartE2EDuration="1.02267745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:30:01.022520746 +0000 UTC m=+1414.709326044" watchObservedRunningTime="2026-09-19 09:30:01.02267745 +0000 UTC m=+1414.709482747"
Sep 19 09:30:01.023056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.023036    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r" podStartSLOduration=1.023028387 podStartE2EDuration="1.023028387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:30:00.999781393 +0000 UTC m=+1414.686586702" watchObservedRunningTime="2026-09-19 09:30:01.023028387 +0000 UTC m=+1414.709833685"
Sep 19 09:30:01.046970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.046865    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt" podStartSLOduration=1.046850183 podStartE2EDuration="1.046850183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:30:01.045354062 +0000 UTC m=+1414.732159359" watchObservedRunningTime="2026-09-19 09:30:01.046850183 +0000 UTC m=+1414.733655559"
Sep 19 09:30:01.993969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.993934    2581 generic.go:356] "Generic (PLEG): container finished" podID="f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301" containerID="37f93a2f0655fa9324a57576b58146f1fbc6fcbeb9f1032b37e12d45adb8e5a0" exitCode=0
Sep 19 09:30:01.994468 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.994013    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt" event={"ID":"f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301","Type":"ContainerDied","Data":"37f93a2f0655fa9324a57576b58146f1fbc6fcbeb9f1032b37e12d45adb8e5a0"}
Sep 19 09:30:01.995587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.995565    2581 generic.go:356] "Generic (PLEG): container finished" podID="7bb58fcb-45fc-44c6-bd06-690de28d1767" containerID="dbebf3a7e5c43edf3c4873f301fb26fb88e7b619a8bdec83bb7a024e70951c19" exitCode=0
Sep 19 09:30:01.995670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.995638    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq" event={"ID":"7bb58fcb-45fc-44c6-bd06-690de28d1767","Type":"ContainerDied","Data":"dbebf3a7e5c43edf3c4873f301fb26fb88e7b619a8bdec83bb7a024e70951c19"}
Sep 19 09:30:01.997134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.997092    2581 generic.go:356] "Generic (PLEG): container finished" podID="d4d66c1c-7ce4-4563-91cd-8226ac101f1e" containerID="eab5c15694c7a02ccb30bf7c3a6d7f45e2a54f80ea7a9dba236dbb20cf4014be" exitCode=0
Sep 19 09:30:01.997226 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:01.997153    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r" event={"ID":"d4d66c1c-7ce4-4563-91cd-8226ac101f1e","Type":"ContainerDied","Data":"eab5c15694c7a02ccb30bf7c3a6d7f45e2a54f80ea7a9dba236dbb20cf4014be"}
Sep 19 09:30:02.329792 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:02.329688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:03.148404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:03.148373    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"
Sep 19 09:30:03.193631 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:03.193596    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"
Sep 19 09:30:03.197004 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:03.196983    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"
Sep 19 09:30:03.469005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:03.468977    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:30:04.008634 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.008607    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"
Sep 19 09:30:04.008799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.008650    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt" event={"ID":"f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301","Type":"ContainerDied","Data":"a708e1ae96fb18e1937e7694cb36a894c11e7ddc2ded760dbfb1e5ffa1ed0e83"}
Sep 19 09:30:04.008799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.008699    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a708e1ae96fb18e1937e7694cb36a894c11e7ddc2ded760dbfb1e5ffa1ed0e83"
Sep 19 09:30:04.009993 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.009975    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"
Sep 19 09:30:04.010140 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.010033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq" event={"ID":"7bb58fcb-45fc-44c6-bd06-690de28d1767","Type":"ContainerDied","Data":"d0b45af07cd27d75389acc9618d76b138b0672940e3913280d26458b68fd8888"}
Sep 19 09:30:04.010140 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.010072    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0b45af07cd27d75389acc9618d76b138b0672940e3913280d26458b68fd8888"
Sep 19 09:30:04.011473 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.011456    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r" event={"ID":"d4d66c1c-7ce4-4563-91cd-8226ac101f1e","Type":"ContainerDied","Data":"f3ddafec6b0a4201ec5b4063dba2ddb594ba8cdc99adcb00effc42b66af8a361"}
Sep 19 09:30:04.011558 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.011486    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3ddafec6b0a4201ec5b4063dba2ddb594ba8cdc99adcb00effc42b66af8a361"
Sep 19 09:30:04.011558 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.011496    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"
Sep 19 09:30:04.091705 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.091666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:04.622364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.622324    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:04.691390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.691361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:04.921648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:04.921614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:05.176757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.176676    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"]
Sep 19 09:30:05.177022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177009    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d66c1c-7ce4-4563-91cd-8226ac101f1e" containerName="cleanup"
Sep 19 09:30:05.177073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177023    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d66c1c-7ce4-4563-91cd-8226ac101f1e" containerName="cleanup"
Sep 19 09:30:05.177073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177036    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301" containerName="cleanup"
Sep 19 09:30:05.177073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177041    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301" containerName="cleanup"
Sep 19 09:30:05.177073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177050    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb58fcb-45fc-44c6-bd06-690de28d1767" containerName="cleanup"
Sep 19 09:30:05.177073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177055    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb58fcb-45fc-44c6-bd06-690de28d1767" containerName="cleanup"
Sep 19 09:30:05.177540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177126    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4d66c1c-7ce4-4563-91cd-8226ac101f1e" containerName="cleanup"
Sep 19 09:30:05.177540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177137    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb58fcb-45fc-44c6-bd06-690de28d1767" containerName="cleanup"
Sep 19 09:30:05.177540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.177143    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301" containerName="cleanup"
Sep 19 09:30:05.180173 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.180156    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.182516 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.182493    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0e698776-openshift-default-dockercfg-46lsd\""
Sep 19 09:30:05.191892 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.191870    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"]
Sep 19 09:30:05.201019 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.200994    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:05.302033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.301989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2wpl\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-kube-api-access-q2wpl\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92a7703-35d3-40b2-bce7-155367675ed9-istio-podinfo\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-data\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92a7703-35d3-40b2-bce7-155367675ed9-istiod-ca-cert\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302260    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-certs\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302293    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-istio-token\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302336    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-socket\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302369    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-envoy\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.302601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.302427    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-credential-socket\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.347845 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.347802    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:05.403791 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.403755    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2wpl\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-kube-api-access-q2wpl\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.403962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.403811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92a7703-35d3-40b2-bce7-155367675ed9-istio-podinfo\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.403962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.403856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-data\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.403962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.403896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92a7703-35d3-40b2-bce7-155367675ed9-istiod-ca-cert\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.403962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.403926    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-certs\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.403962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.403954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-istio-token\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404304 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.403986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-socket\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404304 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.404012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-envoy\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404304 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.404048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-credential-socket\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404467 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.404408    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-data\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.404459    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-certs\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.404497    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-credential-socket\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.404521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-socket\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.404787 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.404765    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92a7703-35d3-40b2-bce7-155367675ed9-istiod-ca-cert\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.406602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.406580    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92a7703-35d3-40b2-bce7-155367675ed9-istio-podinfo\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.406702 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.406650    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-envoy\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.412832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.412761    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-istio-token\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.414726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.414705    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2wpl\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-kube-api-access-q2wpl\") pod \"e2e-mt-0e698776-openshift-default-7699899b-w6mqn\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") " pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.491125 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.491026    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:05.826000 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.825966    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"]
Sep 19 09:30:05.828524 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:30:05.828498    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92a7703_35d3_40b2_bce7_155367675ed9.slice/crio-7d2c4b118fe24e780dffefdd0ceafe185db1907037c942d72f0ef42a6750ca0f WatchSource:0}: Error finding container 7d2c4b118fe24e780dffefdd0ceafe185db1907037c942d72f0ef42a6750ca0f: Status 404 returned error can't find the container with id 7d2c4b118fe24e780dffefdd0ceafe185db1907037c942d72f0ef42a6750ca0f
Sep 19 09:30:05.831284 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.831252    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:30:05.831381 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.831332    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:30:05.831447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:05.831377    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:30:06.025012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:06.024977    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" event={"ID":"c92a7703-35d3-40b2-bce7-155367675ed9","Type":"ContainerStarted","Data":"93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502"}
Sep 19 09:30:06.025210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:06.025033    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" event={"ID":"c92a7703-35d3-40b2-bce7-155367675ed9","Type":"ContainerStarted","Data":"7d2c4b118fe24e780dffefdd0ceafe185db1907037c942d72f0ef42a6750ca0f"}
Sep 19 09:30:06.049837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:06.049774    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podStartSLOduration=1.049751285 podStartE2EDuration="1.049751285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:30:06.047666343 +0000 UTC m=+1419.734471646" watchObservedRunningTime="2026-09-19 09:30:06.049751285 +0000 UTC m=+1419.736556584"
Sep 19 09:30:06.491734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:06.491702    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:06.492991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:06.492956    2581 patch_prober.go:28] interesting pod/e2e-mt-0e698776-openshift-default-7699899b-w6mqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 09:30:06.493153 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:06.493020    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 09:30:07.276005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:07.275964    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:30:07.276460 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:07.276021    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:30:07.288467 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:07.288437    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:30:07.496160 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:07.496132    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:08.035834 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:08.035799    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:08.036976 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:08.036950    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:08.047017 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:08.046999    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:30:10.981483 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:10.981444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:11.663993 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:11.663956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:11.767913 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:11.767879    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:12.064544 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:12.064456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:12.884670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:12.884633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:13.710357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:13.710318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:14.498334 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:14.498296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:15.267257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.267221    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"]
Sep 19 09:30:15.271198 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.271173    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.274459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.274438    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c18110\"/\"kube-root-ca.crt\""
Sep 19 09:30:15.275432 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.275406    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c18110\"/\"test-model-c18110-kserve-self-signed-certs\""
Sep 19 09:30:15.275432 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.275418    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c18110\"/\"openshift-service-ca.crt\""
Sep 19 09:30:15.275604 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.275468    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c18110\"/\"default-dockercfg-bbrd2\""
Sep 19 09:30:15.284551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.284528    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"]
Sep 19 09:30:15.295870 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.295844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-tmp-dir\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.296002 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.295880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-home\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.296002 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.295917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9mzj\" (UniqueName: \"kubernetes.io/projected/348da095-b48d-4ab5-b6e4-b2300c37bafd-kube-api-access-r9mzj\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.296089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.296006    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-dshm\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.296089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.296046    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-model-cache\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.296179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.296136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/348da095-b48d-4ab5-b6e4-b2300c37bafd-tls-certs\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.324894 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.324862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:15.396585 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.396545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-home\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.396770 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.396614    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9mzj\" (UniqueName: \"kubernetes.io/projected/348da095-b48d-4ab5-b6e4-b2300c37bafd-kube-api-access-r9mzj\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.396770 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.396651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-dshm\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.396770 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.396677    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-model-cache\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.396770 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.396734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/348da095-b48d-4ab5-b6e4-b2300c37bafd-tls-certs\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.396945 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.396790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-tmp-dir\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.397057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.397038    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-home\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.397132 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.397081    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-model-cache\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.397222 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.397199    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-tmp-dir\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.399844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.399815    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-dshm\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.400088 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.400073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/348da095-b48d-4ab5-b6e4-b2300c37bafd-tls-certs\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.419296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.419266    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9mzj\" (UniqueName: \"kubernetes.io/projected/348da095-b48d-4ab5-b6e4-b2300c37bafd-kube-api-access-r9mzj\") pod \"test-model-c18110-kserve-67f77d4bf-z5f5w\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") " pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.491056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.491016    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:15.580806 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.580705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:15.668380 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.668340    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:15.859225 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.859199    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"]
Sep 19 09:30:15.860642 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:15.860612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:15.861017 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:30:15.860993    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod348da095_b48d_4ab5_b6e4_b2300c37bafd.slice/crio-3b94015fa1e6153b99660d9025193f313c61fb472b5e8fd439f7aecd006f70fb WatchSource:0}: Error finding container 3b94015fa1e6153b99660d9025193f313c61fb472b5e8fd439f7aecd006f70fb: Status 404 returned error can't find the container with id 3b94015fa1e6153b99660d9025193f313c61fb472b5e8fd439f7aecd006f70fb
Sep 19 09:30:16.006125 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:16.006074    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:16.078803 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:16.078774    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w" event={"ID":"348da095-b48d-4ab5-b6e4-b2300c37bafd","Type":"ContainerStarted","Data":"8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36"}
Sep 19 09:30:16.078977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:16.078814    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w" event={"ID":"348da095-b48d-4ab5-b6e4-b2300c37bafd","Type":"ContainerStarted","Data":"3b94015fa1e6153b99660d9025193f313c61fb472b5e8fd439f7aecd006f70fb"}
Sep 19 09:30:16.100271 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:16.100218    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w" podStartSLOduration=1.100201086 podStartE2EDuration="1.100201086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:30:16.098548754 +0000 UTC m=+1429.785354054" watchObservedRunningTime="2026-09-19 09:30:16.100201086 +0000 UTC m=+1429.787006384"
Sep 19 09:30:16.109546 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:16.109464    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:22.478448 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:22.478415    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:30:25.581079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:25.581038    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:25.581616 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:25.581119    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:25.617979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:25.617947    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:26.140950 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:26.140917    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:30:34.181017 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:34.180977    2581 generic.go:356] "Generic (PLEG): container finished" podID="c92a7703-35d3-40b2-bce7-155367675ed9" containerID="93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502" exitCode=0
Sep 19 09:30:34.181486 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:34.181029    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" event={"ID":"c92a7703-35d3-40b2-bce7-155367675ed9","Type":"ContainerDied","Data":"93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502"}
Sep 19 09:30:34.181534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:34.181507    2581 scope.go:117] "RemoveContainer" containerID="93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502"
Sep 19 09:30:34.182792 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:34.182753    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:30:34.182871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:34.182853    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:30:34.182922 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:34.182906    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:30:35.191916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:35.190979    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" event={"ID":"c92a7703-35d3-40b2-bce7-155367675ed9","Type":"ContainerStarted","Data":"53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe"}
Sep 19 09:30:35.491457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:35.491286    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:35.493653 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:35.493200    2581 patch_prober.go:28] interesting pod/e2e-mt-0e698776-openshift-default-7699899b-w6mqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 09:30:35.493653 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:35.493257    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 09:30:36.491560 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:36.491524    2581 patch_prober.go:28] interesting pod/e2e-mt-0e698776-openshift-default-7699899b-w6mqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 09:30:36.492344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:36.492310    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 09:30:37.491832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:37.491794    2581 patch_prober.go:28] interesting pod/e2e-mt-0e698776-openshift-default-7699899b-w6mqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 09:30:37.492435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:37.492400    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 09:30:37.808081 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:37.807979    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:38.036583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:38.036536    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:38.492203 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:38.492161    2581 patch_prober.go:28] interesting pod/e2e-mt-0e698776-openshift-default-7699899b-w6mqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 09:30:38.492696 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:38.492231    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 09:30:38.530656 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:38.530614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:39.427992 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:39.427951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:39.491976 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:39.491928    2581 patch_prober.go:28] interesting pod/e2e-mt-0e698776-openshift-default-7699899b-w6mqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 19 09:30:39.492213 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:39.492004    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 19 09:30:40.231023 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:40.230989    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:40.618859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:40.618766    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:40.621822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:40.621794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:30:41.098338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:41.098295    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:41.615812 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:41.615780    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:42.038999 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:42.038955    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:42.331774 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:42.331659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:42.436069 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:42.436039    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:42.800258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:42.800226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:48.707990 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.707956    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"]
Sep 19 09:30:48.711533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.711517    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.715294 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.715266    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ac387c\"/\"explicit-model-ac387c-kserve-self-signed-certs\""
Sep 19 09:30:48.715294 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.715285    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ac387c\"/\"default-dockercfg-c99tl\""
Sep 19 09:30:48.715459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.715297    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ac387c\"/\"openshift-service-ca.crt\""
Sep 19 09:30:48.716062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.716041    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ac387c\"/\"kube-root-ca.crt\""
Sep 19 09:30:48.724798 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.724776    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-model-cache\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.724891 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.724817    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tmp-dir\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.724891 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.724884    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-dshm\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.724963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.724913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6hzh\" (UniqueName: \"kubernetes.io/projected/50a9c8cc-fb02-42de-b38b-daf86acf13e1-kube-api-access-z6hzh\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.724963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.724948    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-home\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.725032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.724963    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tls-certs\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.737835 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.737810    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"]
Sep 19 09:30:48.793945 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.793910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:48.825414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tmp-dir\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.825617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825458    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-dshm\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.825617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hzh\" (UniqueName: \"kubernetes.io/projected/50a9c8cc-fb02-42de-b38b-daf86acf13e1-kube-api-access-z6hzh\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.825617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825525    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-home\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.825781 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tls-certs\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.825781 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825753    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-model-cache\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.825884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825798    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tmp-dir\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.825973 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.825953    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-home\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.826092 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.826073    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-model-cache\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.828036 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.828010    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-dshm\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.828540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.828518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tls-certs\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:48.845675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:48.845634    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6hzh\" (UniqueName: \"kubernetes.io/projected/50a9c8cc-fb02-42de-b38b-daf86acf13e1-kube-api-access-z6hzh\") pod \"explicit-model-ac387c-kserve-ddd6768b7-lvnld\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") " pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:49.020738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:49.020641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:49.304617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:49.304586    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"]
Sep 19 09:30:49.305461 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:30:49.305430    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50a9c8cc_fb02_42de_b38b_daf86acf13e1.slice/crio-678f9c31829d60a87216b6ac8b0362b9d0b92dea6dc6d1e83f678a251b75424c WatchSource:0}: Error finding container 678f9c31829d60a87216b6ac8b0362b9d0b92dea6dc6d1e83f678a251b75424c: Status 404 returned error can't find the container with id 678f9c31829d60a87216b6ac8b0362b9d0b92dea6dc6d1e83f678a251b75424c
Sep 19 09:30:49.425968 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:49.425843    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:49.522467 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:49.522433    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:50.283743 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:50.283710    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld" event={"ID":"50a9c8cc-fb02-42de-b38b-daf86acf13e1","Type":"ContainerStarted","Data":"4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01"}
Sep 19 09:30:50.284166 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:50.283756    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld" event={"ID":"50a9c8cc-fb02-42de-b38b-daf86acf13e1","Type":"ContainerStarted","Data":"678f9c31829d60a87216b6ac8b0362b9d0b92dea6dc6d1e83f678a251b75424c"}
Sep 19 09:30:50.304272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:50.304225    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld" podStartSLOduration=2.304206908 podStartE2EDuration="2.304206908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:30:50.30367259 +0000 UTC m=+1463.990477888" watchObservedRunningTime="2026-09-19 09:30:50.304206908 +0000 UTC m=+1463.991012206"
Sep 19 09:30:50.634314 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:50.634282    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:51.352438 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:51.352401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:52.113238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:52.113202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:52.860520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:52.860483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:53.698835 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:53.698798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:54.681261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:54.681224    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:30:54.995972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:54.995891    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:30:59.021668 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:59.021634    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:59.022062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:59.021688    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:59.034235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:59.034206    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:30:59.345837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:30:59.345752    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:31:00.309861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:00.309826    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:00.402072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:00.401993    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:00.479504 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:00.479458    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:00.549685 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:00.549647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:00.627039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:00.627010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:00.900433 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:00.900338    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:00.999699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:00.999664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:01.944336 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:01.944296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:02.649154 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:02.649092    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:03.377880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:03.377845    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:04.111363 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:04.111323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:07.154947 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.154910    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:07.255581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.255549    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:07.404347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.404314    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"]
Sep 19 09:31:07.404608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.404582    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" containerID="cri-o://53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe" gracePeriod=30
Sep 19 09:31:07.449283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.449249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:07.765880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.765784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:07.863173 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.863139    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:07.985784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:07.985748    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:10.621900 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:10.621865    2581 patch_prober.go:28] interesting pod/e2e-mt-0e698776-openshift-default-7699899b-w6mqn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 09:31:10.622311 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:10.621934    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 09:31:11.611534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:11.611494    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:12.655115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.655079    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:31:12.759591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759507    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2wpl\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-kube-api-access-q2wpl\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.759591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759570    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92a7703-35d3-40b2-bce7-155367675ed9-istio-podinfo\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.759790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759601    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-envoy\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.759790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759650    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-data\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.759790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759676    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-credential-socket\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.759790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759710    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-istio-token\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.759790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759739    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92a7703-35d3-40b2-bce7-155367675ed9-istiod-ca-cert\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.760123 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759794    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-socket\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.760123 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759824    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-certs\") pod \"c92a7703-35d3-40b2-bce7-155367675ed9\" (UID: \"c92a7703-35d3-40b2-bce7-155367675ed9\") "
Sep 19 09:31:12.760123 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759960    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-data" (OuterVolumeSpecName: "istio-data") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:12.760123 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.759984    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:12.760350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.760194    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:12.760350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.760207    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-credential-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.760350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.760224    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-data\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.760350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.760230    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:12.760350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.760258    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c92a7703-35d3-40b2-bce7-155367675ed9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:31:12.762157 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.762128    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c92a7703-35d3-40b2-bce7-155367675ed9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 09:31:12.762289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.762174    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-istio-token" (OuterVolumeSpecName: "istio-token") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:31:12.762289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.762194    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:12.762289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.762272    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-kube-api-access-q2wpl" (OuterVolumeSpecName: "kube-api-access-q2wpl") pod "c92a7703-35d3-40b2-bce7-155367675ed9" (UID: "c92a7703-35d3-40b2-bce7-155367675ed9"). InnerVolumeSpecName "kube-api-access-q2wpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:31:12.861515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.861481    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-istio-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.861515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.861511    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c92a7703-35d3-40b2-bce7-155367675ed9-istiod-ca-cert\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.861515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.861521    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.861752 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.861529    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-workload-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.861752 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.861539    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2wpl\" (UniqueName: \"kubernetes.io/projected/c92a7703-35d3-40b2-bce7-155367675ed9-kube-api-access-q2wpl\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.861752 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.861549    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c92a7703-35d3-40b2-bce7-155367675ed9-istio-podinfo\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:12.861752 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:12.861557    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c92a7703-35d3-40b2-bce7-155367675ed9-istio-envoy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:13.410396 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.410356    2581 generic.go:356] "Generic (PLEG): container finished" podID="c92a7703-35d3-40b2-bce7-155367675ed9" containerID="53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe" exitCode=0
Sep 19 09:31:13.410714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.410442    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"
Sep 19 09:31:13.410714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.410449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" event={"ID":"c92a7703-35d3-40b2-bce7-155367675ed9","Type":"ContainerDied","Data":"53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe"}
Sep 19 09:31:13.410714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.410542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn" event={"ID":"c92a7703-35d3-40b2-bce7-155367675ed9","Type":"ContainerDied","Data":"7d2c4b118fe24e780dffefdd0ceafe185db1907037c942d72f0ef42a6750ca0f"}
Sep 19 09:31:13.410714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.410573    2581 scope.go:117] "RemoveContainer" containerID="53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe"
Sep 19 09:31:13.419813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.419795    2581 scope.go:117] "RemoveContainer" containerID="93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502"
Sep 19 09:31:13.430697 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.430674    2581 scope.go:117] "RemoveContainer" containerID="53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe"
Sep 19 09:31:13.431162 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:31:13.431139    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe\": container with ID starting with 53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe not found: ID does not exist" containerID="53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe"
Sep 19 09:31:13.431247 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.431170    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe"} err="failed to get container status \"53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe\": rpc error: code = NotFound desc = could not find container \"53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe\": container with ID starting with 53b22482bb0bf6ac1385d4246345c5041e60692b7e263ebe96909dcebabea5fe not found: ID does not exist"
Sep 19 09:31:13.431247 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.431189    2581 scope.go:117] "RemoveContainer" containerID="93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502"
Sep 19 09:31:13.431746 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:31:13.431727    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502\": container with ID starting with 93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502 not found: ID does not exist" containerID="93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502"
Sep 19 09:31:13.431817 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.431755    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502"} err="failed to get container status \"93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502\": rpc error: code = NotFound desc = could not find container \"93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502\": container with ID starting with 93c742eb3f587956617b738ca488f866d3a3cfa92549f43e48fdc040ea700502 not found: ID does not exist"
Sep 19 09:31:13.433570 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.433549    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"]
Sep 19 09:31:13.435479 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:13.435460    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0e698776-openshift-default-7699899b-w6mqn"]
Sep 19 09:31:14.062346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:14.062314    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:14.209958 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:14.209928    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:14.335296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:14.335208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:14.507799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:14.507765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:14.831348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:14.831316    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" path="/var/lib/kubelet/pods/c92a7703-35d3-40b2-bce7-155367675ed9/volumes"
Sep 19 09:31:16.271664 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:16.271630    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:31:19.413019 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:19.412971    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:20.620652 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:20.620618    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:20.688735 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:20.688696    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:21.493973 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:21.493937    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:21.905784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:21.905737    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:21.986008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:21.985966    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:23.391680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:23.391631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:23.502887 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:23.502847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:24.197364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:24.197324    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:24.757399 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:24.757361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:24.847794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:24.847752    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:25.189192 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:25.189149    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:26.175494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:26.175452    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:27.010686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:27.010650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:27.700252 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:27.700210    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:28.376469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.376435    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"]
Sep 19 09:31:28.376779 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.376765    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy"
Sep 19 09:31:28.376779 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.376778    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy"
Sep 19 09:31:28.376928 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.376789    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy"
Sep 19 09:31:28.376928 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.376794    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy"
Sep 19 09:31:28.376928 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.376869    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy"
Sep 19 09:31:28.376928 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.376879    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c92a7703-35d3-40b2-bce7-155367675ed9" containerName="istio-proxy"
Sep 19 09:31:28.381057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.381033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.383977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.383957    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:31:28.384145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.383957    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 09:31:28.384850 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.384827    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 09:31:28.384964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.384892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-261ce4-serving-cert\""
Sep 19 09:31:28.384964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.384901    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzltl\""
Sep 19 09:31:28.394813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.394789    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"]
Sep 19 09:31:28.403580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.403544    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpq5c\" (UniqueName: \"kubernetes.io/projected/e127e811-65b3-4e6b-beae-51c11489220a-kube-api-access-qpq5c\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.403722 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.403646    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.403784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.403734    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e127e811-65b3-4e6b-beae-51c11489220a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.403832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.403787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.421076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.421043    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:28.504238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.504209    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.504430 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.504257    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e127e811-65b3-4e6b-beae-51c11489220a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.504430 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.504278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.504430 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.504351    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpq5c\" (UniqueName: \"kubernetes.io/projected/e127e811-65b3-4e6b-beae-51c11489220a-kube-api-access-qpq5c\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.505073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.505033    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e127e811-65b3-4e6b-beae-51c11489220a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.507023 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.507003    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.507161 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.507049    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.516873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.516848    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpq5c\" (UniqueName: \"kubernetes.io/projected/e127e811-65b3-4e6b-beae-51c11489220a-kube-api-access-qpq5c\") pod \"maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.522580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.522548    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:28.692734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.692642    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:28.971242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.971217    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"]
Sep 19 09:31:28.973779 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:31:28.973757    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode127e811_65b3_4e6b_beae_51c11489220a.slice/crio-41d12f51cfc02901d0058dbca39063ba139074ffd9142aa320b22e6f2b925ba9 WatchSource:0}: Error finding container 41d12f51cfc02901d0058dbca39063ba139074ffd9142aa320b22e6f2b925ba9: Status 404 returned error can't find the container with id 41d12f51cfc02901d0058dbca39063ba139074ffd9142aa320b22e6f2b925ba9
Sep 19 09:31:28.975398 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:28.975378    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 09:31:29.180627 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:29.180552    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:29.274501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:29.274403    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:29.497964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:29.497928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh" event={"ID":"e127e811-65b3-4e6b-beae-51c11489220a","Type":"ContainerStarted","Data":"2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4"}
Sep 19 09:31:29.498192 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:29.497990    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh" event={"ID":"e127e811-65b3-4e6b-beae-51c11489220a","Type":"ContainerStarted","Data":"41d12f51cfc02901d0058dbca39063ba139074ffd9142aa320b22e6f2b925ba9"}
Sep 19 09:31:29.498192 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:29.498023    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:29.517623 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:29.517573    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh" podStartSLOduration=1.137334286 podStartE2EDuration="1.517556771s" totalImagesPullingTime="380.222485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:31:29.516878588 +0000 UTC m=+1503.203683898" watchObservedRunningTime="2026-09-19 09:31:29.517556771 +0000 UTC m=+1503.204362069"
Sep 19 09:31:29.634591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:29.634561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:30.520385 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:30.520348    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:31.217977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:31.217933    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:35.509189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:35.509159    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:31:36.064782 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.064745    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"]
Sep 19 09:31:36.414484 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.414446    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"]
Sep 19 09:31:36.419657 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.419631    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-ac387c-29830170-6r67r"]
Sep 19 09:31:36.446861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.446827    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"]
Sep 19 09:31:36.447155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.447127    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk" podUID="87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" containerName="payload-pre-processing" containerID="cri-o://0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e" gracePeriod=30
Sep 19 09:31:36.513946 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.513915    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:36.618449 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.618404    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:36.701864 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.701843    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:31:36.715974 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.715941    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:36.785777 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.785744    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-plugins-config-volume\") pod \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") "
Sep 19 09:31:36.785961 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.785822    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjcvh\" (UniqueName: \"kubernetes.io/projected/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-kube-api-access-rjcvh\") pod \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\" (UID: \"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b\") "
Sep 19 09:31:36.786146 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.786125    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" (UID: "87ea1ec1-e542-4f94-a9e3-89a2ff7a249b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:31:36.788190 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.788162    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-kube-api-access-rjcvh" (OuterVolumeSpecName: "kube-api-access-rjcvh") pod "87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" (UID: "87ea1ec1-e542-4f94-a9e3-89a2ff7a249b"). InnerVolumeSpecName "kube-api-access-rjcvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:31:36.831394 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.831361    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4d66c1c-7ce4-4563-91cd-8226ac101f1e" path="/var/lib/kubelet/pods/d4d66c1c-7ce4-4563-91cd-8226ac101f1e/volumes"
Sep 19 09:31:36.886805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.886762    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:36.886805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.886794    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjcvh\" (UniqueName: \"kubernetes.io/projected/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b-kube-api-access-rjcvh\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:36.986746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:36.986650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:37.052327 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.052291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:37.545087 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.545050    2581 generic.go:356] "Generic (PLEG): container finished" podID="87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" containerID="0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e" exitCode=0
Sep 19 09:31:37.545551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.545119    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"
Sep 19 09:31:37.545551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.545144    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk" event={"ID":"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b","Type":"ContainerDied","Data":"0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e"}
Sep 19 09:31:37.545551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.545198    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk" event={"ID":"87ea1ec1-e542-4f94-a9e3-89a2ff7a249b","Type":"ContainerDied","Data":"8ff95731dbd9f873f446d69c1761654fe7e1efe649e119187d2933baf42bcaa5"}
Sep 19 09:31:37.545551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.545222    2581 scope.go:117] "RemoveContainer" containerID="0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e"
Sep 19 09:31:37.554191 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.554173    2581 scope.go:117] "RemoveContainer" containerID="0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e"
Sep 19 09:31:37.554494 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:31:37.554474    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e\": container with ID starting with 0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e not found: ID does not exist" containerID="0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e"
Sep 19 09:31:37.554545 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.554502    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e"} err="failed to get container status \"0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e\": rpc error: code = NotFound desc = could not find container \"0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e\": container with ID starting with 0d5d48794f4507a9a1ba76b96ec691fc860e4af7facb0883d1ca9d7d7e686c7e not found: ID does not exist"
Sep 19 09:31:37.561458 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.561436    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"]
Sep 19 09:31:37.570674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.570648    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ac387c-546677dff4-gsnhk"]
Sep 19 09:31:37.781851 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:37.781804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:38.591671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:38.591640    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:31:38.609689 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:38.609656    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:38.831255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:38.831224    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" path="/var/lib/kubelet/pods/87ea1ec1-e542-4f94-a9e3-89a2ff7a249b/volumes"
Sep 19 09:31:39.622815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:39.622776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:40.730910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:40.730875    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:41.724471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:41.724430    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:42.111020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:42.110922    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:42.849528 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:42.849491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:43.821572 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:43.821534    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:44.449701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:44.449658    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:44.757454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:44.757352    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:44.858972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:44.858938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:45.099264 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:45.099156    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:45.554293 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:45.554251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:45.840901 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:45.840797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:46.747918 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:46.747847    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:47.719685 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:47.719585    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:48.188292 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:48.188246    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:48.294286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:48.294251    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:48.723594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:48.723561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:49.873490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:49.873460    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:50.911574 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:50.911523    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:51.084905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.084863    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld" podUID="50a9c8cc-fb02-42de-b38b-daf86acf13e1" containerName="main" containerID="cri-o://4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01" gracePeriod=45
Sep 19 09:31:51.344437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.344411    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:31:51.424396 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424363    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-home\") pod \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") "
Sep 19 09:31:51.424588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424414    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tmp-dir\") pod \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") "
Sep 19 09:31:51.424588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424429    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-dshm\") pod \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") "
Sep 19 09:31:51.424588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424506    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-model-cache\") pod \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") "
Sep 19 09:31:51.424588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424539    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6hzh\" (UniqueName: \"kubernetes.io/projected/50a9c8cc-fb02-42de-b38b-daf86acf13e1-kube-api-access-z6hzh\") pod \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") "
Sep 19 09:31:51.424588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424562    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tls-certs\") pod \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\" (UID: \"50a9c8cc-fb02-42de-b38b-daf86acf13e1\") "
Sep 19 09:31:51.424826 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424677    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-home" (OuterVolumeSpecName: "home") pod "50a9c8cc-fb02-42de-b38b-daf86acf13e1" (UID: "50a9c8cc-fb02-42de-b38b-daf86acf13e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:51.424826 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "50a9c8cc-fb02-42de-b38b-daf86acf13e1" (UID: "50a9c8cc-fb02-42de-b38b-daf86acf13e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:51.424826 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424760    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-model-cache" (OuterVolumeSpecName: "model-cache") pod "50a9c8cc-fb02-42de-b38b-daf86acf13e1" (UID: "50a9c8cc-fb02-42de-b38b-daf86acf13e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:51.424985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424974    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:51.425038 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.424990    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:51.425038 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.425003    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:51.426932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.426907    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "50a9c8cc-fb02-42de-b38b-daf86acf13e1" (UID: "50a9c8cc-fb02-42de-b38b-daf86acf13e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:31:51.426932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.426913    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-dshm" (OuterVolumeSpecName: "dshm") pod "50a9c8cc-fb02-42de-b38b-daf86acf13e1" (UID: "50a9c8cc-fb02-42de-b38b-daf86acf13e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:31:51.426932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.426923    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50a9c8cc-fb02-42de-b38b-daf86acf13e1-kube-api-access-z6hzh" (OuterVolumeSpecName: "kube-api-access-z6hzh") pod "50a9c8cc-fb02-42de-b38b-daf86acf13e1" (UID: "50a9c8cc-fb02-42de-b38b-daf86acf13e1"). InnerVolumeSpecName "kube-api-access-z6hzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:31:51.525844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.525760    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6hzh\" (UniqueName: \"kubernetes.io/projected/50a9c8cc-fb02-42de-b38b-daf86acf13e1-kube-api-access-z6hzh\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:51.525844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.525789    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50a9c8cc-fb02-42de-b38b-daf86acf13e1-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:51.525844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.525800    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50a9c8cc-fb02-42de-b38b-daf86acf13e1-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:31:51.627377 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.627341    2581 generic.go:356] "Generic (PLEG): container finished" podID="50a9c8cc-fb02-42de-b38b-daf86acf13e1" containerID="4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01" exitCode=0
Sep 19 09:31:51.627549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.627392    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"
Sep 19 09:31:51.627549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.627426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld" event={"ID":"50a9c8cc-fb02-42de-b38b-daf86acf13e1","Type":"ContainerDied","Data":"4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01"}
Sep 19 09:31:51.627549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.627475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld" event={"ID":"50a9c8cc-fb02-42de-b38b-daf86acf13e1","Type":"ContainerDied","Data":"678f9c31829d60a87216b6ac8b0362b9d0b92dea6dc6d1e83f678a251b75424c"}
Sep 19 09:31:51.627549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.627495    2581 scope.go:117] "RemoveContainer" containerID="4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01"
Sep 19 09:31:51.637017 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.636999    2581 scope.go:117] "RemoveContainer" containerID="4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01"
Sep 19 09:31:51.637339 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:31:51.637319    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01\": container with ID starting with 4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01 not found: ID does not exist" containerID="4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01"
Sep 19 09:31:51.637409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.637349    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01"} err="failed to get container status \"4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01\": rpc error: code = NotFound desc = could not find container \"4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01\": container with ID starting with 4c6cf52ad518ff5c25f8efea7f10daa7db205c2306b359bcd07fe04fdbe5de01 not found: ID does not exist"
Sep 19 09:31:51.649357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.649331    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"]
Sep 19 09:31:51.652944 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.652922    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-ac387c/explicit-model-ac387c-kserve-ddd6768b7-lvnld"]
Sep 19 09:31:51.739439 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:51.739400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:52.727083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.727045    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"]
Sep 19 09:31:52.727583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.727433    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" containerName="payload-pre-processing"
Sep 19 09:31:52.727583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.727447    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" containerName="payload-pre-processing"
Sep 19 09:31:52.727583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.727468    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50a9c8cc-fb02-42de-b38b-daf86acf13e1" containerName="main"
Sep 19 09:31:52.727583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.727473    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="50a9c8cc-fb02-42de-b38b-daf86acf13e1" containerName="main"
Sep 19 09:31:52.727583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.727537    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="50a9c8cc-fb02-42de-b38b-daf86acf13e1" containerName="main"
Sep 19 09:31:52.727583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.727546    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="87ea1ec1-e542-4f94-a9e3-89a2ff7a249b" containerName="payload-pre-processing"
Sep 19 09:31:52.731935 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.731914    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.735065 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.735046    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-66c0fb-serving-cert\""
Sep 19 09:31:52.754370 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.754335    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"]
Sep 19 09:31:52.784252 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.784208    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:52.832004 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.831972    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50a9c8cc-fb02-42de-b38b-daf86acf13e1" path="/var/lib/kubelet/pods/50a9c8cc-fb02-42de-b38b-daf86acf13e1/volumes"
Sep 19 09:31:52.837912 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.837887    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.838058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.837921    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5qcl\" (UniqueName: \"kubernetes.io/projected/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-kube-api-access-z5qcl\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.838058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.837946    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.838058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.837975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.892354 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.892320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:52.939468 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.939434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.939621 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.939477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5qcl\" (UniqueName: \"kubernetes.io/projected/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-kube-api-access-z5qcl\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.939621 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.939499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.939706 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:31:52.939617    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-serving-cert: secret "maas-api-e2e-shared-b-w2-66c0fb-serving-cert" not found
Sep 19 09:31:52.939706 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:31:52.939698    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls podName:d3c9e0b3-544d-4631-aaa2-77e88bc45d05 nodeName:}" failed. No retries permitted until 2026-09-19 09:31:53.439674917 +0000 UTC m=+1527.126480194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls") pod "maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8" (UID: "d3c9e0b3-544d-4631-aaa2-77e88bc45d05") : secret "maas-api-e2e-shared-b-w2-66c0fb-serving-cert" not found
Sep 19 09:31:52.939820 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.939757    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.940500 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.940480    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.942234 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.942216    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:52.949541 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:52.949517    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5qcl\" (UniqueName: \"kubernetes.io/projected/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-kube-api-access-z5qcl\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:53.140349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.140311    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:53.340819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.340786    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:53.441252 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.441166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:53.445418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.445394    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:53.448044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.448024    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:53.642362 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.642331    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:53.912740 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.912705    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:31:53.927470 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:53.927444    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"]
Sep 19 09:31:53.929420 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:31:53.929394    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c9e0b3_544d_4631_aaa2_77e88bc45d05.slice/crio-9ddd7751b2dcf91c6dc537c927d2f42375fb17113e1a2bcd3b17fe8073351ba3 WatchSource:0}: Error finding container 9ddd7751b2dcf91c6dc537c927d2f42375fb17113e1a2bcd3b17fe8073351ba3: Status 404 returned error can't find the container with id 9ddd7751b2dcf91c6dc537c927d2f42375fb17113e1a2bcd3b17fe8073351ba3
Sep 19 09:31:54.649593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:54.649560    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8" event={"ID":"d3c9e0b3-544d-4631-aaa2-77e88bc45d05","Type":"ContainerStarted","Data":"04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e"}
Sep 19 09:31:54.649785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:54.649615    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8" event={"ID":"d3c9e0b3-544d-4631-aaa2-77e88bc45d05","Type":"ContainerStarted","Data":"9ddd7751b2dcf91c6dc537c927d2f42375fb17113e1a2bcd3b17fe8073351ba3"}
Sep 19 09:31:54.649785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:54.649703    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:31:54.669285 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:54.669237    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8" podStartSLOduration=2.341558532 podStartE2EDuration="2.669223413s" totalImagesPullingTime="327.664881ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:31:54.668235547 +0000 UTC m=+1528.355040858" watchObservedRunningTime="2026-09-19 09:31:54.669223413 +0000 UTC m=+1528.356028711"
Sep 19 09:31:54.806749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:31:54.806716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:00.662088 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:00.662059    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:32:07.680763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.680723    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"]
Sep 19 09:32:07.684937 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.684918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.687373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.687348    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"distinct-w0-14e3f6-kserve-self-signed-certs\""
Sep 19 09:32:07.694313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.694288    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"]
Sep 19 09:32:07.715839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.715769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:07.783972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.783936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-tmp-dir\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.784221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.784021    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-dshm\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.784221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.784056    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzthr\" (UniqueName: \"kubernetes.io/projected/844f2cc3-0f3a-4c0b-9015-149a1f935650-kube-api-access-gzthr\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.784221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.784093    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/844f2cc3-0f3a-4c0b-9015-149a1f935650-tls-certs\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.784221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.784180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-model-cache\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.784221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.784208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-home\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.884880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.884831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-dshm\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.884880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.884877    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzthr\" (UniqueName: \"kubernetes.io/projected/844f2cc3-0f3a-4c0b-9015-149a1f935650-kube-api-access-gzthr\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.885206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.885005    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/844f2cc3-0f3a-4c0b-9015-149a1f935650-tls-certs\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.885206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.885065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-model-cache\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.885206 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.885086    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-home\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.885364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.885217    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-tmp-dir\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.885538 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.885506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-model-cache\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.885793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.885569    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-home\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.885793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.885635    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-tmp-dir\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.887501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.887472    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-dshm\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.887745 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.887728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/844f2cc3-0f3a-4c0b-9015-149a1f935650-tls-certs\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.894115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.894077    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzthr\" (UniqueName: \"kubernetes.io/projected/844f2cc3-0f3a-4c0b-9015-149a1f935650-kube-api-access-gzthr\") pod \"distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:07.995353 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:07.995252    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:08.270009 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:08.269984    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"]
Sep 19 09:32:08.272808 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:32:08.272782    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod844f2cc3_0f3a_4c0b_9015_149a1f935650.slice/crio-8335d6df9a91e7441c7dde5f648ac145c060b8d268e158c35a8f122d8795c14e WatchSource:0}: Error finding container 8335d6df9a91e7441c7dde5f648ac145c060b8d268e158c35a8f122d8795c14e: Status 404 returned error can't find the container with id 8335d6df9a91e7441c7dde5f648ac145c060b8d268e158c35a8f122d8795c14e
Sep 19 09:32:08.381538 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:08.381499    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:32:08.728953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:08.728914    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf" event={"ID":"844f2cc3-0f3a-4c0b-9015-149a1f935650","Type":"ContainerStarted","Data":"9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba"}
Sep 19 09:32:08.729364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:08.728967    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf" event={"ID":"844f2cc3-0f3a-4c0b-9015-149a1f935650","Type":"ContainerStarted","Data":"8335d6df9a91e7441c7dde5f648ac145c060b8d268e158c35a8f122d8795c14e"}
Sep 19 09:32:08.751410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:08.751360    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf" podStartSLOduration=1.751343634 podStartE2EDuration="1.751343634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:32:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:32:08.748648692 +0000 UTC m=+1542.435453990" watchObservedRunningTime="2026-09-19 09:32:08.751343634 +0000 UTC m=+1542.438148931"
Sep 19 09:32:13.389630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:13.389593    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:15.320308 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:15.320255    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:16.047129 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:16.047081    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:16.813656 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:16.813607    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:17.618969 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:17.618938    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:17.995399 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:17.995366    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:17.995794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:17.995418    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:18.008376 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:18.008349    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:18.191953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:18.191916    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:18.356025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:18.355917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:18.793039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:18.793011    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:32:19.525665 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:19.525631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:20.098124 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:20.098059    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:20.296784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:20.296749    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"]
Sep 19 09:32:20.297001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:20.296954    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy" containerID="cri-o://2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62" gracePeriod=30
Sep 19 09:32:20.321889 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:20.321852    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:20.730257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:20.730221    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:21.062467 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:21.062358    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:21.409952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:21.409919    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:22.534899 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:22.534856    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:22.637897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:22.637865    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:22.745412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:22.745372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:25.552150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.552123    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:32:25.665345 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665303    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-socket\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665369    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-data\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665417    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istiod-ca-cert\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665486    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-certs\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665530    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-token\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665553    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xds7\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-kube-api-access-4xds7\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665586    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-podinfo\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665598    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:32:25.665730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665631    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-credential-socket\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.665730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665687    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:32:25.665730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665698    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-envoy\") pod \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\" (UID: \"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7\") "
Sep 19 09:32:25.666039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:32:25.666039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665859    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:32:25.666039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.665871    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:32:25.666249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.666065    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.666249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.666084    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-data\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.666249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.666116    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istiod-ca-cert\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.666249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.666130    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-workload-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.666249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.666141    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-credential-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.668039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.668007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:32:25.668194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.668037    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:32:25.668194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.668036    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 09:32:25.668297 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.668271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-kube-api-access-4xds7" (OuterVolumeSpecName: "kube-api-access-4xds7") pod "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" (UID: "f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7"). InnerVolumeSpecName "kube-api-access-4xds7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:32:25.767432 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.767388    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.767432 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.767421    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xds7\" (UniqueName: \"kubernetes.io/projected/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-kube-api-access-4xds7\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.767432 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.767432    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-podinfo\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.767432 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.767441    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7-istio-envoy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:25.820257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.820223    2581 generic.go:356] "Generic (PLEG): container finished" podID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerID="2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62" exitCode=0
Sep 19 09:32:25.820390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.820267    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" event={"ID":"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7","Type":"ContainerDied","Data":"2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62"}
Sep 19 09:32:25.820390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.820273    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"
Sep 19 09:32:25.820390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.820309    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl" event={"ID":"f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7","Type":"ContainerDied","Data":"58e3d1f34aa5fde8d2ee5c0b53574dc122db5379f5370245a25c213093557d23"}
Sep 19 09:32:25.820390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.820332    2581 scope.go:117] "RemoveContainer" containerID="2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62"
Sep 19 09:32:25.830184 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.830164    2581 scope.go:117] "RemoveContainer" containerID="a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947"
Sep 19 09:32:25.839243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.839214    2581 scope.go:117] "RemoveContainer" containerID="2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62"
Sep 19 09:32:25.839570 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:32:25.839548    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62\": container with ID starting with 2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62 not found: ID does not exist" containerID="2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62"
Sep 19 09:32:25.839640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.839581    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62"} err="failed to get container status \"2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62\": rpc error: code = NotFound desc = could not find container \"2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62\": container with ID starting with 2ce9963a581e2f05ae1ce6754f9c9d0e8075cf81002e151a6173497d19471a62 not found: ID does not exist"
Sep 19 09:32:25.839640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.839598    2581 scope.go:117] "RemoveContainer" containerID="a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947"
Sep 19 09:32:25.839866 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:32:25.839849    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947\": container with ID starting with a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947 not found: ID does not exist" containerID="a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947"
Sep 19 09:32:25.839908 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.839872    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947"} err="failed to get container status \"a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947\": rpc error: code = NotFound desc = could not find container \"a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947\": container with ID starting with a522bdad9e51caeb26ad6249506a715df1e9eeb077e93cfae32fe08f908e3947 not found: ID does not exist"
Sep 19 09:32:25.842820 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.842798    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"]
Sep 19 09:32:25.847851 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:25.847826    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-ac387c-openshift-default-596677fd56-pl2dl"]
Sep 19 09:32:26.830809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.830780    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" path="/var/lib/kubelet/pods/f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7/volumes"
Sep 19 09:32:26.965816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.965781    2581 scope.go:117] "RemoveContainer" containerID="dcff64f6c63f8736952168dd3bd8a571dbf0120cfc7cf078ec8cb77647f025a6"
Sep 19 09:32:26.966090 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.966069    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7845-9f2d75"]
Sep 19 09:32:26.966527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.966505    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy"
Sep 19 09:32:26.966527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.966522    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy"
Sep 19 09:32:26.966527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.966531    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy"
Sep 19 09:32:26.966715 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.966537    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy"
Sep 19 09:32:26.966715 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.966608    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy"
Sep 19 09:32:26.966715 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.966618    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2b09d7b-2b5d-4978-89dc-7a7050fbfeb7" containerName="istio-proxy"
Sep 19 09:32:26.969833 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.969815    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-9f2d75"
Sep 19 09:32:26.972346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.972324    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q5mhq\""
Sep 19 09:32:26.974377 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:26.974352    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-9f2d75"]
Sep 19 09:32:27.077748 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.077720    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76rlm\" (UniqueName: \"kubernetes.io/projected/e09c6599-a96d-4e0a-b891-45d3035240c3-kube-api-access-76rlm\") pod \"test-curl-7845-9f2d75\" (UID: \"e09c6599-a96d-4e0a-b891-45d3035240c3\") " pod="opendatahub/test-curl-7845-9f2d75"
Sep 19 09:32:27.178826 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.178790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76rlm\" (UniqueName: \"kubernetes.io/projected/e09c6599-a96d-4e0a-b891-45d3035240c3-kube-api-access-76rlm\") pod \"test-curl-7845-9f2d75\" (UID: \"e09c6599-a96d-4e0a-b891-45d3035240c3\") " pod="opendatahub/test-curl-7845-9f2d75"
Sep 19 09:32:27.187824 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.187793    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76rlm\" (UniqueName: \"kubernetes.io/projected/e09c6599-a96d-4e0a-b891-45d3035240c3-kube-api-access-76rlm\") pod \"test-curl-7845-9f2d75\" (UID: \"e09c6599-a96d-4e0a-b891-45d3035240c3\") " pod="opendatahub/test-curl-7845-9f2d75"
Sep 19 09:32:27.281396 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.281368    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-9f2d75"
Sep 19 09:32:27.557348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.557267    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-9f2d75"]
Sep 19 09:32:27.560703 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:32:27.560673    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode09c6599_a96d_4e0a_b891_45d3035240c3.slice/crio-125dd1f82da75e6e34cfadae9df1ccdb1a4000f9f68072254baf453667c9c6f9 WatchSource:0}: Error finding container 125dd1f82da75e6e34cfadae9df1ccdb1a4000f9f68072254baf453667c9c6f9: Status 404 returned error can't find the container with id 125dd1f82da75e6e34cfadae9df1ccdb1a4000f9f68072254baf453667c9c6f9
Sep 19 09:32:27.833360 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.833258    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-9f2d75" event={"ID":"e09c6599-a96d-4e0a-b891-45d3035240c3","Type":"ContainerStarted","Data":"090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a"}
Sep 19 09:32:27.833360 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.833303    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-9f2d75" event={"ID":"e09c6599-a96d-4e0a-b891-45d3035240c3","Type":"ContainerStarted","Data":"125dd1f82da75e6e34cfadae9df1ccdb1a4000f9f68072254baf453667c9c6f9"}
Sep 19 09:32:27.848717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:27.848666    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7845-9f2d75" podStartSLOduration=1.8486496909999999 podStartE2EDuration="1.848649691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:32:27.847459834 +0000 UTC m=+1561.534265132" watchObservedRunningTime="2026-09-19 09:32:27.848649691 +0000 UTC m=+1561.535454989"
Sep 19 09:32:28.451079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:28.451039    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7845-9f2d75"]
Sep 19 09:32:28.456958 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:28.456930    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7845-9f2d75"]
Sep 19 09:32:28.838476 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:32:28.838446    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7845-9f2d75" container="test-curl-7845-9f2d75"
Sep 19 09:32:29.043739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.043709    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:29.328751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.328716    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"]
Sep 19 09:32:29.329175 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.329157    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09c6599-a96d-4e0a-b891-45d3035240c3" containerName="test-curl-7845-9f2d75"
Sep 19 09:32:29.329279 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.329177    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09c6599-a96d-4e0a-b891-45d3035240c3" containerName="test-curl-7845-9f2d75"
Sep 19 09:32:29.329279 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.329258    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e09c6599-a96d-4e0a-b891-45d3035240c3" containerName="test-curl-7845-9f2d75"
Sep 19 09:32:29.332419 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.332397    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.336456 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.336434    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 09:32:29.336803 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.336784    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e46994f4-openshift-default-dockercfg-n6zqc\""
Sep 19 09:32:29.349515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.349489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"]
Sep 19 09:32:29.378644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.378611    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:29.399382 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399349    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-envoy\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399499 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399384    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-data\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399499 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399413    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-socket\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399574 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-podinfo\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399607 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399569    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-credential-socket\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399607 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399600    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-token\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399631    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zppxs\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-kube-api-access-zppxs\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399669    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-certs\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.399738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.399704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istiod-ca-cert\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.500704 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.500666    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zppxs\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-kube-api-access-zppxs\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.500865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.500710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-certs\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.500865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.500762    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istiod-ca-cert\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.500865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.500794    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-envoy\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.500865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.500817    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-data\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.500964    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-socket\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501043    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-podinfo\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501198 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501181    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-credential-socket\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501218    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-token\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501229    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-certs\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501350 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501255    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-data\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501419 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501400    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-socket\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501583    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-credential-socket\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.501799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.501782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istiod-ca-cert\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.503431 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.503409    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-envoy\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.503761 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.503742    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-podinfo\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.510415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.510391    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-token\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.510582 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.510564    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zppxs\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-kube-api-access-zppxs\") pod \"e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") " pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.642932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.642896    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:29.843059 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.843014    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7845-9f2d75" podUID="e09c6599-a96d-4e0a-b891-45d3035240c3" containerName="test-curl-7845-9f2d75" containerID="cri-o://090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a" gracePeriod=2
Sep 19 09:32:29.894918 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.894823    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:29.926566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.926541    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"]
Sep 19 09:32:29.927442 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:32:29.927418    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bb2ea4_256a_40c7_be55_7bda3cdf81a7.slice/crio-f71761300ce65818a79d1e14c059ba2c399f9cdbcf812b9b668e28fa43bdcac8 WatchSource:0}: Error finding container f71761300ce65818a79d1e14c059ba2c399f9cdbcf812b9b668e28fa43bdcac8: Status 404 returned error can't find the container with id f71761300ce65818a79d1e14c059ba2c399f9cdbcf812b9b668e28fa43bdcac8
Sep 19 09:32:29.930330 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.930296    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:32:29.930460 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.930382    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:32:29.930460 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:29.930436    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:32:30.849186 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:30.849148    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" event={"ID":"00bb2ea4-256a-40c7-be55-7bda3cdf81a7","Type":"ContainerStarted","Data":"3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84"}
Sep 19 09:32:30.849666 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:30.849203    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" event={"ID":"00bb2ea4-256a-40c7-be55-7bda3cdf81a7","Type":"ContainerStarted","Data":"f71761300ce65818a79d1e14c059ba2c399f9cdbcf812b9b668e28fa43bdcac8"}
Sep 19 09:32:30.883374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:30.883310    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podStartSLOduration=1.883286316 podStartE2EDuration="1.883286316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:32:30.883089306 +0000 UTC m=+1564.569894607" watchObservedRunningTime="2026-09-19 09:32:30.883286316 +0000 UTC m=+1564.570091615"
Sep 19 09:32:31.643957 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:31.643905    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:31.649250 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:31.649224    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:31.854675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:31.854648    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:31.855858 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:31.855836    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:32.017393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.017370    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-9f2d75"
Sep 19 09:32:32.024198 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.024166    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:32:32.128908 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.128876    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76rlm\" (UniqueName: \"kubernetes.io/projected/e09c6599-a96d-4e0a-b891-45d3035240c3-kube-api-access-76rlm\") pod \"e09c6599-a96d-4e0a-b891-45d3035240c3\" (UID: \"e09c6599-a96d-4e0a-b891-45d3035240c3\") "
Sep 19 09:32:32.131257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.131223    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09c6599-a96d-4e0a-b891-45d3035240c3-kube-api-access-76rlm" (OuterVolumeSpecName: "kube-api-access-76rlm") pod "e09c6599-a96d-4e0a-b891-45d3035240c3" (UID: "e09c6599-a96d-4e0a-b891-45d3035240c3"). InnerVolumeSpecName "kube-api-access-76rlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:32:32.134343 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.134297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:32.230176 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.230069    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76rlm\" (UniqueName: \"kubernetes.io/projected/e09c6599-a96d-4e0a-b891-45d3035240c3-kube-api-access-76rlm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:32:32.804855 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.804820    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:32.830861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.830827    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e09c6599-a96d-4e0a-b891-45d3035240c3" path="/var/lib/kubelet/pods/e09c6599-a96d-4e0a-b891-45d3035240c3/volumes"
Sep 19 09:32:32.862128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.861085    2581 generic.go:356] "Generic (PLEG): container finished" podID="e09c6599-a96d-4e0a-b891-45d3035240c3" containerID="090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a" exitCode=137
Sep 19 09:32:32.862128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.861922    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-9f2d75"
Sep 19 09:32:32.862632 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.862254    2581 scope.go:117] "RemoveContainer" containerID="090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a"
Sep 19 09:32:32.872663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.872644    2581 scope.go:117] "RemoveContainer" containerID="090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a"
Sep 19 09:32:32.872945 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:32:32.872921    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a\": container with ID starting with 090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a not found: ID does not exist" containerID="090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a"
Sep 19 09:32:32.873025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.872953    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a"} err="failed to get container status \"090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a\": rpc error: code = NotFound desc = could not find container \"090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a\": container with ID starting with 090d0e601ae6c5afe5b1fe368a494e765704bc7c815ab8144a515c818d6af24a not found: ID does not exist"
Sep 19 09:32:32.915447 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:32.915398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:33.132561 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:33.132522    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:34.101962 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:34.101898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:34.809751 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:34.809716    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:35.648454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:35.648417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:36.433720 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:36.433689    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:37.435322 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:37.435278    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:37.559482 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:37.559443    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:37.966170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:37.966132    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:38.068288 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:38.068253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:54.417671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:54.417629    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:54.518539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:54.518447    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:57.334758 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:57.334717    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:58.008009 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:58.007972    2581 generic.go:356] "Generic (PLEG): container finished" podID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerID="3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84" exitCode=0
Sep 19 09:32:58.008202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:58.008069    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" event={"ID":"00bb2ea4-256a-40c7-be55-7bda3cdf81a7","Type":"ContainerDied","Data":"3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84"}
Sep 19 09:32:58.008510 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:58.008495    2581 scope.go:117] "RemoveContainer" containerID="3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84"
Sep 19 09:32:58.009799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:58.009762    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:32:58.009897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:58.009845    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:32:58.009897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:58.009877    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:32:59.015639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:59.015599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" event={"ID":"00bb2ea4-256a-40c7-be55-7bda3cdf81a7","Type":"ContainerStarted","Data":"a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078"}
Sep 19 09:32:59.276808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:59.276699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:32:59.643824 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:59.643790    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:32:59.645150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:59.645124    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 19 09:32:59.645270 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:32:59.645185    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 19 09:33:00.643725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:00.643687    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 19 09:33:00.644144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:00.643770    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 19 09:33:00.831037 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:00.831003    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:01.597607 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:01.597565    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:01.644148 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:01.644089    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 19 09:33:01.644526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:01.644170    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 19 09:33:01.699089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:01.699028    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:33:01.855319 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:01.855237    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:33:02.533220 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:02.533188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:02.644355 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:02.644308    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 19 09:33:02.644836 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:02.644390    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 19 09:33:02.924914 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:02.924882    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:03.643425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:03.643384    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 19 09:33:03.643622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:03.643445    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 19 09:33:03.656014 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:03.655976    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:04.420344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:04.420291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:04.643335 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:04.643297    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 19 09:33:04.643544 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:04.643377    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 19 09:33:05.252952 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:05.252888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:05.643475 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:05.643437    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 19 09:33:05.643670 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:05.643496    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 19 09:33:05.989247 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:05.989151    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:06.648053 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:06.648023    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:33:06.648996 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:06.648980    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:33:10.205633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.205602    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"]
Sep 19 09:33:10.209378 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.209360    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.212806 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.212780    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-e200e8\"/\"openshift-service-ca.crt\""
Sep 19 09:33:10.212929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.212780    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-e200e8\"/\"noten-model-e200e8-kserve-self-signed-certs\""
Sep 19 09:33:10.212929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.212817    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-e200e8\"/\"kube-root-ca.crt\""
Sep 19 09:33:10.212929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.212785    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-e200e8\"/\"default-dockercfg-qhhkj\""
Sep 19 09:33:10.219970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.219946    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"]
Sep 19 09:33:10.234323 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.234300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:10.292213 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.292181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-model-cache\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.292347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.292238    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdlj9\" (UniqueName: \"kubernetes.io/projected/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-kube-api-access-cdlj9\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.292347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.292295    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-dshm\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.292347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.292325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-home\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.292459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.292365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tls-certs\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.292459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.292446    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tmp-dir\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.393742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.393704    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdlj9\" (UniqueName: \"kubernetes.io/projected/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-kube-api-access-cdlj9\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.393925 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.393747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-dshm\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.393925 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.393879    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-home\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.393925 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.393918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tls-certs\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.394114 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.393972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tmp-dir\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.394114 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.394065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-model-cache\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.394306 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.394283    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-home\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.394414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.394385    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tmp-dir\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.394414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.394405    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-model-cache\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.396183 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.396157    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-dshm\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.396677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.396660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tls-certs\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.402450 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.402415    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdlj9\" (UniqueName: \"kubernetes.io/projected/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-kube-api-access-cdlj9\") pod \"noten-model-e200e8-kserve-79c94c84c5-k2s95\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") " pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.519730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.519643    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:10.794990 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:10.794960    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"]
Sep 19 09:33:10.797046 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:33:10.797004    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf71b9ef_cccb_49f6_a70c_3c9f317e75b1.slice/crio-6fe6f21598d3863daf6e775a8566254ee41546db3821f4ecc3aa4feb04267c7f WatchSource:0}: Error finding container 6fe6f21598d3863daf6e775a8566254ee41546db3821f4ecc3aa4feb04267c7f: Status 404 returned error can't find the container with id 6fe6f21598d3863daf6e775a8566254ee41546db3821f4ecc3aa4feb04267c7f
Sep 19 09:33:11.077924 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:11.077829    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95" event={"ID":"af71b9ef-cccb-49f6-a70c-3c9f317e75b1","Type":"ContainerStarted","Data":"dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434"}
Sep 19 09:33:11.077924 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:11.077870    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95" event={"ID":"af71b9ef-cccb-49f6-a70c-3c9f317e75b1","Type":"ContainerStarted","Data":"6fe6f21598d3863daf6e775a8566254ee41546db3821f4ecc3aa4feb04267c7f"}
Sep 19 09:33:11.094657 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:11.094598    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95" podStartSLOduration=1.09457856 podStartE2EDuration="1.09457856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:33:11.094076262 +0000 UTC m=+1604.780881567" watchObservedRunningTime="2026-09-19 09:33:11.09457856 +0000 UTC m=+1604.781383860"
Sep 19 09:33:12.656499 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:12.656463    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"]
Sep 19 09:33:12.675211 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:12.675166    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:14.589933 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:14.589897    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"]
Sep 19 09:33:14.660566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:14.660526    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:14.980508 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:14.980479    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:15.700958 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:15.700918    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:16.396459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:16.396423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:17.159401 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:17.159366    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:17.905179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:17.905149    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:18.807342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:18.807302    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:19.501854 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:19.501812    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:20.200270 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:20.200233    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:20.520317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:20.520231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:20.520317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:20.520281    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:20.532750 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:20.532723    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:21.141295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:21.141269    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:33:25.329147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.329112    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:25.370919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.370878    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"]
Sep 19 09:33:25.371168 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.371141    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8" podUID="2df89df4-5099-4763-8b9b-beef4f2e101d" containerName="payload-pre-processing" containerID="cri-o://475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8" gracePeriod=30
Sep 19 09:33:25.430754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.430722    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:25.529441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.529400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:25.632143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.632076    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:25.633761 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.633744    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:33:25.637899 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.637871    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df89df4-5099-4763-8b9b-beef4f2e101d-plugins-config-volume\") pod \"2df89df4-5099-4763-8b9b-beef4f2e101d\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") "
Sep 19 09:33:25.638038 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.637952    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx47n\" (UniqueName: \"kubernetes.io/projected/2df89df4-5099-4763-8b9b-beef4f2e101d-kube-api-access-wx47n\") pod \"2df89df4-5099-4763-8b9b-beef4f2e101d\" (UID: \"2df89df4-5099-4763-8b9b-beef4f2e101d\") "
Sep 19 09:33:25.638306 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.638282    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2df89df4-5099-4763-8b9b-beef4f2e101d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2df89df4-5099-4763-8b9b-beef4f2e101d" (UID: "2df89df4-5099-4763-8b9b-beef4f2e101d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:33:25.640273 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.640249    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df89df4-5099-4763-8b9b-beef4f2e101d-kube-api-access-wx47n" (OuterVolumeSpecName: "kube-api-access-wx47n") pod "2df89df4-5099-4763-8b9b-beef4f2e101d" (UID: "2df89df4-5099-4763-8b9b-beef4f2e101d"). InnerVolumeSpecName "kube-api-access-wx47n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:33:25.738863 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.738822    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df89df4-5099-4763-8b9b-beef4f2e101d-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:25.738863 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.738854    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx47n\" (UniqueName: \"kubernetes.io/projected/2df89df4-5099-4763-8b9b-beef4f2e101d-kube-api-access-wx47n\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:25.904894 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:25.904854    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:26.008168 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.008091    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:26.158051 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.157959    2581 generic.go:356] "Generic (PLEG): container finished" podID="2df89df4-5099-4763-8b9b-beef4f2e101d" containerID="475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8" exitCode=0
Sep 19 09:33:26.158051 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.158018    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"
Sep 19 09:33:26.158330 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.158050    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8" event={"ID":"2df89df4-5099-4763-8b9b-beef4f2e101d","Type":"ContainerDied","Data":"475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8"}
Sep 19 09:33:26.158330 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.158133    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8" event={"ID":"2df89df4-5099-4763-8b9b-beef4f2e101d","Type":"ContainerDied","Data":"c2d92653feba0a1afec2d2595e0322c5263b02e1e59522f5b25dfea4a4936907"}
Sep 19 09:33:26.158330 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.158167    2581 scope.go:117] "RemoveContainer" containerID="475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8"
Sep 19 09:33:26.167753 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.167734    2581 scope.go:117] "RemoveContainer" containerID="475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8"
Sep 19 09:33:26.168137 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:33:26.168086    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8\": container with ID starting with 475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8 not found: ID does not exist" containerID="475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8"
Sep 19 09:33:26.168242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.168148    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8"} err="failed to get container status \"475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8\": rpc error: code = NotFound desc = could not find container \"475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8\": container with ID starting with 475b012769901d3c32bf51b805c7752302a32325ddc668bccd0e4d8aa9de66d8 not found: ID does not exist"
Sep 19 09:33:26.182662 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.182627    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"]
Sep 19 09:33:26.188491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.188462    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8"]
Sep 19 09:33:26.623575 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.623476    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9dfb0f-68977f4c9d-zbsg8" podUID="2df89df4-5099-4763-8b9b-beef4f2e101d" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.35:9004: i/o timeout"
Sep 19 09:33:26.648896 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.648854    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:26.832142 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:26.832085    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2df89df4-5099-4763-8b9b-beef4f2e101d" path="/var/lib/kubelet/pods/2df89df4-5099-4763-8b9b-beef4f2e101d/volumes"
Sep 19 09:33:27.339853 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.339760    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:27.676896 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.676856    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn" podUID="84910cf6-750e-4958-90fe-3a2febec319f" containerName="main" containerID="cri-o://4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea" gracePeriod=45
Sep 19 09:33:27.929861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.929789    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:33:27.955719 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.955680    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-model-cache\") pod \"84910cf6-750e-4958-90fe-3a2febec319f\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") "
Sep 19 09:33:27.955875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.955739    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84910cf6-750e-4958-90fe-3a2febec319f-tls-certs\") pod \"84910cf6-750e-4958-90fe-3a2febec319f\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") "
Sep 19 09:33:27.955875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.955777    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-dshm\") pod \"84910cf6-750e-4958-90fe-3a2febec319f\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") "
Sep 19 09:33:27.955875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.955824    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7prmb\" (UniqueName: \"kubernetes.io/projected/84910cf6-750e-4958-90fe-3a2febec319f-kube-api-access-7prmb\") pod \"84910cf6-750e-4958-90fe-3a2febec319f\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") "
Sep 19 09:33:27.955875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.955867    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-home\") pod \"84910cf6-750e-4958-90fe-3a2febec319f\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") "
Sep 19 09:33:27.956070 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.955887    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-tmp-dir\") pod \"84910cf6-750e-4958-90fe-3a2febec319f\" (UID: \"84910cf6-750e-4958-90fe-3a2febec319f\") "
Sep 19 09:33:27.956070 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.955991    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-model-cache" (OuterVolumeSpecName: "model-cache") pod "84910cf6-750e-4958-90fe-3a2febec319f" (UID: "84910cf6-750e-4958-90fe-3a2febec319f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:27.956230 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.956207    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-home" (OuterVolumeSpecName: "home") pod "84910cf6-750e-4958-90fe-3a2febec319f" (UID: "84910cf6-750e-4958-90fe-3a2febec319f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:27.956283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.956230    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "84910cf6-750e-4958-90fe-3a2febec319f" (UID: "84910cf6-750e-4958-90fe-3a2febec319f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:27.956341 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.956333    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:27.956404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.956348    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:27.956404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.956363    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:27.958217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.958190    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-dshm" (OuterVolumeSpecName: "dshm") pod "84910cf6-750e-4958-90fe-3a2febec319f" (UID: "84910cf6-750e-4958-90fe-3a2febec319f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:27.958217 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.958212    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84910cf6-750e-4958-90fe-3a2febec319f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "84910cf6-750e-4958-90fe-3a2febec319f" (UID: "84910cf6-750e-4958-90fe-3a2febec319f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:33:27.958357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:27.958271    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84910cf6-750e-4958-90fe-3a2febec319f-kube-api-access-7prmb" (OuterVolumeSpecName: "kube-api-access-7prmb") pod "84910cf6-750e-4958-90fe-3a2febec319f" (UID: "84910cf6-750e-4958-90fe-3a2febec319f"). InnerVolumeSpecName "kube-api-access-7prmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:33:28.056847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.056810    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84910cf6-750e-4958-90fe-3a2febec319f-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:28.056847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.056837    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84910cf6-750e-4958-90fe-3a2febec319f-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:28.056847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.056846    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7prmb\" (UniqueName: \"kubernetes.io/projected/84910cf6-750e-4958-90fe-3a2febec319f-kube-api-access-7prmb\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:28.115956 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.115917    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:28.169983 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.169945    2581 generic.go:356] "Generic (PLEG): container finished" podID="84910cf6-750e-4958-90fe-3a2febec319f" containerID="4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea" exitCode=0
Sep 19 09:33:28.170171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.169991    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"
Sep 19 09:33:28.170171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.170020    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn" event={"ID":"84910cf6-750e-4958-90fe-3a2febec319f","Type":"ContainerDied","Data":"4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea"}
Sep 19 09:33:28.170171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.170059    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn" event={"ID":"84910cf6-750e-4958-90fe-3a2febec319f","Type":"ContainerDied","Data":"f3e3134a30564a562e45dc2c8ade1bb150b46e7dbb79328fa1d80f90beacee0f"}
Sep 19 09:33:28.170171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.170078    2581 scope.go:117] "RemoveContainer" containerID="4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea"
Sep 19 09:33:28.180191 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.180157    2581 scope.go:117] "RemoveContainer" containerID="4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea"
Sep 19 09:33:28.180498 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:33:28.180471    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea\": container with ID starting with 4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea not found: ID does not exist" containerID="4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea"
Sep 19 09:33:28.180600 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.180501    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea"} err="failed to get container status \"4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea\": rpc error: code = NotFound desc = could not find container \"4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea\": container with ID starting with 4ea9441cf32529ade93798ef4e3a81a2da1ad7e25c79626c0b8031ca76d55cea not found: ID does not exist"
Sep 19 09:33:28.191685 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.191653    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"]
Sep 19 09:33:28.195715 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.195694    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9dfb0f/test-model-9dfb0f-kserve-5f5d8f6875-h4sfn"]
Sep 19 09:33:28.799299 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.799263    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:28.831813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:28.831780    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84910cf6-750e-4958-90fe-3a2febec319f" path="/var/lib/kubelet/pods/84910cf6-750e-4958-90fe-3a2febec319f/volumes"
Sep 19 09:33:29.476027 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.475994    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:29.609481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.609434    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w" podUID="348da095-b48d-4ab5-b6e4-b2300c37bafd" containerName="main" containerID="cri-o://8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36" gracePeriod=45
Sep 19 09:33:29.858044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.858023    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:33:29.870347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870320    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-home\") pod \"348da095-b48d-4ab5-b6e4-b2300c37bafd\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") "
Sep 19 09:33:29.870347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870348    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-dshm\") pod \"348da095-b48d-4ab5-b6e4-b2300c37bafd\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") "
Sep 19 09:33:29.870554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-tmp-dir\") pod \"348da095-b48d-4ab5-b6e4-b2300c37bafd\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") "
Sep 19 09:33:29.870554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870418    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-model-cache\") pod \"348da095-b48d-4ab5-b6e4-b2300c37bafd\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") "
Sep 19 09:33:29.870554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870440    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/348da095-b48d-4ab5-b6e4-b2300c37bafd-tls-certs\") pod \"348da095-b48d-4ab5-b6e4-b2300c37bafd\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") "
Sep 19 09:33:29.870554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870511    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9mzj\" (UniqueName: \"kubernetes.io/projected/348da095-b48d-4ab5-b6e4-b2300c37bafd-kube-api-access-r9mzj\") pod \"348da095-b48d-4ab5-b6e4-b2300c37bafd\" (UID: \"348da095-b48d-4ab5-b6e4-b2300c37bafd\") "
Sep 19 09:33:29.870761 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870658    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-home" (OuterVolumeSpecName: "home") pod "348da095-b48d-4ab5-b6e4-b2300c37bafd" (UID: "348da095-b48d-4ab5-b6e4-b2300c37bafd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:29.870761 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870674    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "348da095-b48d-4ab5-b6e4-b2300c37bafd" (UID: "348da095-b48d-4ab5-b6e4-b2300c37bafd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:29.870761 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870726    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-model-cache" (OuterVolumeSpecName: "model-cache") pod "348da095-b48d-4ab5-b6e4-b2300c37bafd" (UID: "348da095-b48d-4ab5-b6e4-b2300c37bafd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:29.870907 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870843    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:29.870907 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870860    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:29.870907 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.870875    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:29.872709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.872686    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-dshm" (OuterVolumeSpecName: "dshm") pod "348da095-b48d-4ab5-b6e4-b2300c37bafd" (UID: "348da095-b48d-4ab5-b6e4-b2300c37bafd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:33:29.872813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.872792    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/348da095-b48d-4ab5-b6e4-b2300c37bafd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "348da095-b48d-4ab5-b6e4-b2300c37bafd" (UID: "348da095-b48d-4ab5-b6e4-b2300c37bafd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:33:29.872915 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.872898    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/348da095-b48d-4ab5-b6e4-b2300c37bafd-kube-api-access-r9mzj" (OuterVolumeSpecName: "kube-api-access-r9mzj") pod "348da095-b48d-4ab5-b6e4-b2300c37bafd" (UID: "348da095-b48d-4ab5-b6e4-b2300c37bafd"). InnerVolumeSpecName "kube-api-access-r9mzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:33:29.971335 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.971300    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/348da095-b48d-4ab5-b6e4-b2300c37bafd-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:29.971335 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.971332    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9mzj\" (UniqueName: \"kubernetes.io/projected/348da095-b48d-4ab5-b6e4-b2300c37bafd-kube-api-access-r9mzj\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:29.971335 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:29.971341    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/348da095-b48d-4ab5-b6e4-b2300c37bafd-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:33:30.182759 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.182719    2581 generic.go:356] "Generic (PLEG): container finished" podID="348da095-b48d-4ab5-b6e4-b2300c37bafd" containerID="8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36" exitCode=0
Sep 19 09:33:30.182966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.182770    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w" event={"ID":"348da095-b48d-4ab5-b6e4-b2300c37bafd","Type":"ContainerDied","Data":"8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36"}
Sep 19 09:33:30.182966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.182779    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"
Sep 19 09:33:30.182966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.182804    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w" event={"ID":"348da095-b48d-4ab5-b6e4-b2300c37bafd","Type":"ContainerDied","Data":"3b94015fa1e6153b99660d9025193f313c61fb472b5e8fd439f7aecd006f70fb"}
Sep 19 09:33:30.182966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.182820    2581 scope.go:117] "RemoveContainer" containerID="8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36"
Sep 19 09:33:30.192389 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.192303    2581 scope.go:117] "RemoveContainer" containerID="8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36"
Sep 19 09:33:30.195239 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:33:30.195207    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36\": container with ID starting with 8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36 not found: ID does not exist" containerID="8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36"
Sep 19 09:33:30.195353 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.195256    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36"} err="failed to get container status \"8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36\": rpc error: code = NotFound desc = could not find container \"8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36\": container with ID starting with 8df979d709627bc40b0839cb380f9ff628aa69de8de3ad7fb338c0144a021f36 not found: ID does not exist"
Sep 19 09:33:30.205414 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.205390    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"]
Sep 19 09:33:30.209500 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.209476    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c18110/test-model-c18110-kserve-67f77d4bf-z5f5w"]
Sep 19 09:33:30.831575 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:30.831539    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="348da095-b48d-4ab5-b6e4-b2300c37bafd" path="/var/lib/kubelet/pods/348da095-b48d-4ab5-b6e4-b2300c37bafd/volumes"
Sep 19 09:33:31.183232 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:31.183204    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:33:32.428365 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:32.428325    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:32.506743 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:32.506704    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:32.603343 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:32.603307    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:32.815993 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:32.815900    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:33.223713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:33.223681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:33.973786 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:33.973748    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:34.890316 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:34.890281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:35.582472 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:35.582419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:36.246457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:36.246413    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:37.103327 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:37.103290    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:37.796956 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:37.796926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:43.607426 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:43.607397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:43.616652 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:43.616625    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"]
Sep 19 09:33:43.618475 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:43.618448    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c18110-29830170-mr5xq"]
Sep 19 09:33:43.680643 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:43.680609    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:43.744552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:43.744520    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:43.817889 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:43.817855    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:43.882485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:43.882441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:44.118018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:44.117986    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:44.219858 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:44.219768    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:44.823680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:44.823647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:44.831146 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:44.831121    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb58fcb-45fc-44c6-bd06-690de28d1767" path="/var/lib/kubelet/pods/7bb58fcb-45fc-44c6-bd06-690de28d1767/volumes"
Sep 19 09:33:45.517504 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:45.517473    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:45.774067 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:45.773976    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:45.960886 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:45.960841    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:46.066659 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:46.066566    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:46.617419 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:46.617380    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:47.427778 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:47.427736    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:49.971815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:49.971782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:50.671629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:50.671591    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:33:51.047880 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:51.047793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:51.050867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:51.050841    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"]
Sep 19 09:33:51.115655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:51.115614    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:51.557599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:51.557553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:52.065609 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.065574    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"]
Sep 19 09:33:52.065981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.065965    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df89df4-5099-4763-8b9b-beef4f2e101d" containerName="payload-pre-processing"
Sep 19 09:33:52.065981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.065978    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df89df4-5099-4763-8b9b-beef4f2e101d" containerName="payload-pre-processing"
Sep 19 09:33:52.066073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.065996    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84910cf6-750e-4958-90fe-3a2febec319f" containerName="main"
Sep 19 09:33:52.066073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.066001    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="84910cf6-750e-4958-90fe-3a2febec319f" containerName="main"
Sep 19 09:33:52.066073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.066014    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="348da095-b48d-4ab5-b6e4-b2300c37bafd" containerName="main"
Sep 19 09:33:52.066073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.066019    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="348da095-b48d-4ab5-b6e4-b2300c37bafd" containerName="main"
Sep 19 09:33:52.066073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.066073    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="348da095-b48d-4ab5-b6e4-b2300c37bafd" containerName="main"
Sep 19 09:33:52.066255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.066081    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="84910cf6-750e-4958-90fe-3a2febec319f" containerName="main"
Sep 19 09:33:52.066255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.066088    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df89df4-5099-4763-8b9b-beef4f2e101d" containerName="payload-pre-processing"
Sep 19 09:33:52.070408 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.070390    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.073371 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.073350    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"distinct-2-w0-14e3f6-kserve-self-signed-certs\""
Sep 19 09:33:52.081545 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.081518    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"]
Sep 19 09:33:52.178861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.178829    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-home\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.179049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.178903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tmp-dir\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.179049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.178932    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-dshm\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.179049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.178971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tls-certs\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.179049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.179024    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpwcn\" (UniqueName: \"kubernetes.io/projected/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-kube-api-access-kpwcn\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.179275 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.179077    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-model-cache\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.268397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.268361    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:52.280049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280021    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-home\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280219 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tmp-dir\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280219 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280087    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-dshm\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280219 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280138    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tls-certs\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280219 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280169    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpwcn\" (UniqueName: \"kubernetes.io/projected/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-kube-api-access-kpwcn\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280219 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-model-cache\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280504    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-home\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280521    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tmp-dir\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.280635 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.280584    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-model-cache\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.282721 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.282700    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-dshm\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.283015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.282994    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tls-certs\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.288633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.288609    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpwcn\" (UniqueName: \"kubernetes.io/projected/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-kube-api-access-kpwcn\") pod \"distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") " pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.379742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.379700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:33:52.654908 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:52.654878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"]
Sep 19 09:33:52.656772 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:33:52.656745    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod340bfca3_6480_43e8_9dcb_2eb7765fb1dc.slice/crio-5d2db1c9072eb2cab62e028c44089d6fb412d8e13807133895c2640c036f2a17 WatchSource:0}: Error finding container 5d2db1c9072eb2cab62e028c44089d6fb412d8e13807133895c2640c036f2a17: Status 404 returned error can't find the container with id 5d2db1c9072eb2cab62e028c44089d6fb412d8e13807133895c2640c036f2a17
Sep 19 09:33:53.102650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:53.102540    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:53.313522 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:53.313492    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67" event={"ID":"340bfca3-6480-43e8-9dcb-2eb7765fb1dc","Type":"ContainerStarted","Data":"49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd"}
Sep 19 09:33:53.313522 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:53.313529    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67" event={"ID":"340bfca3-6480-43e8-9dcb-2eb7765fb1dc","Type":"ContainerStarted","Data":"5d2db1c9072eb2cab62e028c44089d6fb412d8e13807133895c2640c036f2a17"}
Sep 19 09:33:53.332014 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:53.331962    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67" podStartSLOduration=1.331944839 podStartE2EDuration="1.331944839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:33:53.330737139 +0000 UTC m=+1647.017542451" watchObservedRunningTime="2026-09-19 09:33:53.331944839 +0000 UTC m=+1647.018750170"
Sep 19 09:33:53.704810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:53.704744    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:53.796259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:53.796226    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:54.033492 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:54.033383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:54.783338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:54.783300    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:55.750149 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:55.750094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:56.605703 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:56.605661    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:57.407709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:57.407677    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:57.461567 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:57.461530    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:57.780553 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:57.780467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:57.971346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:57.971312    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:33:58.069435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:33:58.069339    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:01.348938 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:01.348898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:01.449942 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:01.449911    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:02.380758 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:02.380719    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:34:02.381296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:02.380775    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:34:02.393342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:02.393316    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:34:03.375944 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:03.375908    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:34:06.070639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.070592    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95" podUID="af71b9ef-cccb-49f6-a70c-3c9f317e75b1" containerName="main" containerID="cri-o://dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434" gracePeriod=45
Sep 19 09:34:06.319147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.319121    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:34:06.384855 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.384820    2581 generic.go:356] "Generic (PLEG): container finished" podID="af71b9ef-cccb-49f6-a70c-3c9f317e75b1" containerID="dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434" exitCode=0
Sep 19 09:34:06.385057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.384872    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"
Sep 19 09:34:06.385057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.384907    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95" event={"ID":"af71b9ef-cccb-49f6-a70c-3c9f317e75b1","Type":"ContainerDied","Data":"dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434"}
Sep 19 09:34:06.385057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.384966    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95" event={"ID":"af71b9ef-cccb-49f6-a70c-3c9f317e75b1","Type":"ContainerDied","Data":"6fe6f21598d3863daf6e775a8566254ee41546db3821f4ecc3aa4feb04267c7f"}
Sep 19 09:34:06.385057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.384990    2581 scope.go:117] "RemoveContainer" containerID="dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434"
Sep 19 09:34:06.395069 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.395043    2581 scope.go:117] "RemoveContainer" containerID="dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434"
Sep 19 09:34:06.395408 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:34:06.395389    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434\": container with ID starting with dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434 not found: ID does not exist" containerID="dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434"
Sep 19 09:34:06.395478 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.395417    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434"} err="failed to get container status \"dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434\": rpc error: code = NotFound desc = could not find container \"dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434\": container with ID starting with dd4f183bf237fb5ff886fbb81723c9c5bfdb511fd2e3612fc4174a609b6b7434 not found: ID does not exist"
Sep 19 09:34:06.415356 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-home\") pod \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") "
Sep 19 09:34:06.415505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415364    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-dshm\") pod \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") "
Sep 19 09:34:06.415505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415423    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-model-cache\") pod \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") "
Sep 19 09:34:06.415505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415472    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdlj9\" (UniqueName: \"kubernetes.io/projected/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-kube-api-access-cdlj9\") pod \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") "
Sep 19 09:34:06.415505 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415496    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tmp-dir\") pod \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") "
Sep 19 09:34:06.415711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415531    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tls-certs\") pod \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\" (UID: \"af71b9ef-cccb-49f6-a70c-3c9f317e75b1\") "
Sep 19 09:34:06.415711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415623    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-home" (OuterVolumeSpecName: "home") pod "af71b9ef-cccb-49f6-a70c-3c9f317e75b1" (UID: "af71b9ef-cccb-49f6-a70c-3c9f317e75b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:34:06.415815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415747    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-model-cache" (OuterVolumeSpecName: "model-cache") pod "af71b9ef-cccb-49f6-a70c-3c9f317e75b1" (UID: "af71b9ef-cccb-49f6-a70c-3c9f317e75b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:34:06.415881 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415847    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af71b9ef-cccb-49f6-a70c-3c9f317e75b1" (UID: "af71b9ef-cccb-49f6-a70c-3c9f317e75b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:34:06.415975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415956    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:34:06.416026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415979    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:34:06.416026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.415994    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:34:06.417816 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.417786    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-dshm" (OuterVolumeSpecName: "dshm") pod "af71b9ef-cccb-49f6-a70c-3c9f317e75b1" (UID: "af71b9ef-cccb-49f6-a70c-3c9f317e75b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:34:06.417979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.417825    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af71b9ef-cccb-49f6-a70c-3c9f317e75b1" (UID: "af71b9ef-cccb-49f6-a70c-3c9f317e75b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:34:06.417979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.417830    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-kube-api-access-cdlj9" (OuterVolumeSpecName: "kube-api-access-cdlj9") pod "af71b9ef-cccb-49f6-a70c-3c9f317e75b1" (UID: "af71b9ef-cccb-49f6-a70c-3c9f317e75b1"). InnerVolumeSpecName "kube-api-access-cdlj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:34:06.517263 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.517223    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdlj9\" (UniqueName: \"kubernetes.io/projected/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-kube-api-access-cdlj9\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:34:06.517263 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.517256    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:34:06.517263 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.517269    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af71b9ef-cccb-49f6-a70c-3c9f317e75b1-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:34:06.707458 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.707428    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"]
Sep 19 09:34:06.710811 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.710786    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-e200e8/noten-model-e200e8-kserve-79c94c84c5-k2s95"]
Sep 19 09:34:06.831425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:06.831393    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af71b9ef-cccb-49f6-a70c-3c9f317e75b1" path="/var/lib/kubelet/pods/af71b9ef-cccb-49f6-a70c-3c9f317e75b1/volumes"
Sep 19 09:34:19.638797 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:19.638750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:20.086120 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:20.086017    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:34:20.539189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:20.539126    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:21.632569 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:21.632533    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:22.372349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:22.372314    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:23.159775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.159743    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:23.782283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.782249    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:23.889029 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.888992    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:23.974501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.974467    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"]
Sep 19 09:34:23.975022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.975007    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af71b9ef-cccb-49f6-a70c-3c9f317e75b1" containerName="main"
Sep 19 09:34:23.975073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.975026    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="af71b9ef-cccb-49f6-a70c-3c9f317e75b1" containerName="main"
Sep 19 09:34:23.975150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.975139    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="af71b9ef-cccb-49f6-a70c-3c9f317e75b1" containerName="main"
Sep 19 09:34:23.979695 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.979671    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:23.985303 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.985277    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e46994f4\"/\"kube-root-ca.crt\""
Sep 19 09:34:23.985555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.985310    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e46994f4\"/\"openshift-service-ca.crt\""
Sep 19 09:34:23.985555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.985310    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e46994f4\"/\"default-dockercfg-xntgv\""
Sep 19 09:34:23.985741 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.985372    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e46994f4\"/\"e2e-lifecycle-model-e46994f4-kserve-self-signed-certs\""
Sep 19 09:34:23.991136 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:23.991111    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"]
Sep 19 09:34:24.022418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.022382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:24.059809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.059719    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-tmp-dir\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.059809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.059757    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-home\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.060018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.059838    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwjjb\" (UniqueName: \"kubernetes.io/projected/06d60781-d5e8-4acf-9e0a-967978d5b306-kube-api-access-gwjjb\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.060018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.059871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-model-cache\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.060018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.059893    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-dshm\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.060018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.059983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d60781-d5e8-4acf-9e0a-967978d5b306-tls-certs\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.122737 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.122699    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:24.161461 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.161428    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwjjb\" (UniqueName: \"kubernetes.io/projected/06d60781-d5e8-4acf-9e0a-967978d5b306-kube-api-access-gwjjb\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.161461 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.161467    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-model-cache\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.162026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.161497    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-dshm\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.162026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.161556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d60781-d5e8-4acf-9e0a-967978d5b306-tls-certs\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.162026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.161643    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-tmp-dir\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.162026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.161667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-home\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.162026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.161923    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-model-cache\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.162026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.162004    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-tmp-dir\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.162295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.162036    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-home\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.164057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.164040    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-dshm\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.164256 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.164239    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d60781-d5e8-4acf-9e0a-967978d5b306-tls-certs\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.171159 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.171142    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwjjb\" (UniqueName: \"kubernetes.io/projected/06d60781-d5e8-4acf-9e0a-967978d5b306-kube-api-access-gwjjb\") pod \"e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") " pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.289846 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.289804    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:24.441538 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.441503    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:24.537630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.537594    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:24.566837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:24.566813    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"]
Sep 19 09:34:24.568610 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:34:24.568578    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06d60781_d5e8_4acf_9e0a_967978d5b306.slice/crio-5343deaf9ac80865109a5d2f34492d2a324ebd04c52180c604675a35e0a6eb5a WatchSource:0}: Error finding container 5343deaf9ac80865109a5d2f34492d2a324ebd04c52180c604675a35e0a6eb5a: Status 404 returned error can't find the container with id 5343deaf9ac80865109a5d2f34492d2a324ebd04c52180c604675a35e0a6eb5a
Sep 19 09:34:25.482810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:25.482777    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82" event={"ID":"06d60781-d5e8-4acf-9e0a-967978d5b306","Type":"ContainerStarted","Data":"d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e"}
Sep 19 09:34:25.482810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:25.482817    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82" event={"ID":"06d60781-d5e8-4acf-9e0a-967978d5b306","Type":"ContainerStarted","Data":"5343deaf9ac80865109a5d2f34492d2a324ebd04c52180c604675a35e0a6eb5a"}
Sep 19 09:34:25.501683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:25.501628    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82" podStartSLOduration=2.501610172 podStartE2EDuration="2.501610172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:34:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:34:25.499437159 +0000 UTC m=+1679.186242460" watchObservedRunningTime="2026-09-19 09:34:25.501610172 +0000 UTC m=+1679.188415470"
Sep 19 09:34:27.238785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:27.238749    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:27.335631 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:27.335535    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:27.954910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:27.954860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:28.042839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:28.042801    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:28.145420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:28.145387    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:28.979633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:28.979595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:29.145143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:29.145111    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:29.877585 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:29.877547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:30.713020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:30.712980    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:31.568756 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:31.568706    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:32.373171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:32.373140    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:32.474841 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:32.474803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:32.860196 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:32.860154    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:33.026666 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.026631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:33.120433 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.120354    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"]
Sep 19 09:34:33.122522 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.122490    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:33.125141 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.125125    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.127851 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.127830    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-1afa13\""
Sep 19 09:34:33.127964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.127881    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-1afa13-dockercfg-856vq\""
Sep 19 09:34:33.135309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.135283    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"]
Sep 19 09:34:33.251965 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.251924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.252150 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.251977    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vml6r\" (UniqueName: \"kubernetes.io/projected/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-kube-api-access-vml6r\") pod \"payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.353601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.353559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.353783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.353618    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vml6r\" (UniqueName: \"kubernetes.io/projected/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-kube-api-access-vml6r\") pod \"payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.354358 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.354339    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.362943 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.362915    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vml6r\" (UniqueName: \"kubernetes.io/projected/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-kube-api-access-vml6r\") pod \"payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.435740 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.435705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:33.714190 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:33.714165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"]
Sep 19 09:34:33.716540 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:34:33.716514    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76a4fc5_4a86_4a21_bc6d_1ff073968f99.slice/crio-33205dc6046e18a457bab820ce64176f966c2c49dc18f4c78695da77c8d8f97a WatchSource:0}: Error finding container 33205dc6046e18a457bab820ce64176f966c2c49dc18f4c78695da77c8d8f97a: Status 404 returned error can't find the container with id 33205dc6046e18a457bab820ce64176f966c2c49dc18f4c78695da77c8d8f97a
Sep 19 09:34:34.290294 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.290249    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:34.290506 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.290320    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:34.305397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.305360    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:34.533196 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.533156    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8" event={"ID":"a76a4fc5-4a86-4a21-bc6d-1ff073968f99","Type":"ContainerStarted","Data":"dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e"}
Sep 19 09:34:34.533651 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.533260    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8" event={"ID":"a76a4fc5-4a86-4a21-bc6d-1ff073968f99","Type":"ContainerStarted","Data":"33205dc6046e18a457bab820ce64176f966c2c49dc18f4c78695da77c8d8f97a"}
Sep 19 09:34:34.533651 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.533461    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:34.544610 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.544535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:34:34.552606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:34.552558    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8" podStartSLOduration=1.552542917 podStartE2EDuration="1.552542917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:34:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:34:34.550708107 +0000 UTC m=+1688.237513409" watchObservedRunningTime="2026-09-19 09:34:34.552542917 +0000 UTC m=+1688.239348214"
Sep 19 09:34:35.272455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:35.272417    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:34:37.530459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:37.530419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:37.627685 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:37.627650    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:45.541978 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:45.541941    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:34:54.703626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:54.703541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:55.517007 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:55.516969    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:56.207091 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:56.207053    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:34:56.387406 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:56.387360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:57.333181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:57.333136    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:58.012674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:58.012619    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:58.117650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:58.117615    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:59.070517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:59.070482    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:34:59.919367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:34:59.919326    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:13.634842 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:13.634798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:13.735950 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:13.735905    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:21.469016 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:21.468974    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:35:25.696033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:25.695992    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:26.496650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.496610    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:26.869003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.868922    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"]
Sep 19 09:35:26.872444 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.872424    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:26.876794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.876761    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3240ec\"/\"auth-test-model-3240ec-kserve-self-signed-certs\""
Sep 19 09:35:26.879194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.877616    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3240ec\"/\"openshift-service-ca.crt\""
Sep 19 09:35:26.880987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.880796    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3240ec\"/\"default-dockercfg-vczpg\""
Sep 19 09:35:26.881063 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.881026    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-3240ec\"/\"kube-root-ca.crt\""
Sep 19 09:35:26.886220 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.886195    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"]
Sep 19 09:35:26.905801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.905771    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:26.970476 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.970429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-dshm\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:26.970682 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.970486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-tmp-dir\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:26.970682 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.970546    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-home\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:26.970682 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.970581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-model-cache\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:26.970871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.970727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95df44b9-143b-4f44-8510-85739e72fb74-tls-certs\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:26.970871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:26.970768    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h695x\" (UniqueName: \"kubernetes.io/projected/95df44b9-143b-4f44-8510-85739e72fb74-kube-api-access-h695x\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.043317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.043286    2581 scope.go:117] "RemoveContainer" containerID="4626bf6dd6073ab019d02ef0b3d3dfe72e7cf34c500f39bcfc0a8fef6b75459f"
Sep 19 09:35:27.071852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.071820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-dshm\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.071975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.071863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-tmp-dir\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.071975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.071908    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-home\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.071975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.071932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-model-cache\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.072170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.071984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95df44b9-143b-4f44-8510-85739e72fb74-tls-certs\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.072170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.072019    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h695x\" (UniqueName: \"kubernetes.io/projected/95df44b9-143b-4f44-8510-85739e72fb74-kube-api-access-h695x\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.072327 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.072306    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-tmp-dir\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.072391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.072328    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-model-cache\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.072391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.072376    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-home\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.074266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.074236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-dshm\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.074563 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.074544    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95df44b9-143b-4f44-8510-85739e72fb74-tls-certs\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.080386 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.080363    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h695x\" (UniqueName: \"kubernetes.io/projected/95df44b9-143b-4f44-8510-85739e72fb74-kube-api-access-h695x\") pod \"auth-test-model-3240ec-kserve-589f79c8c-rjkqw\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.184449 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.184417    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:27.213649 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.213612    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:27.463899 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.463873    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"]
Sep 19 09:35:27.466846 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:35:27.466814    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95df44b9_143b_4f44_8510_85739e72fb74.slice/crio-ed2c90ae4cdb165483962a7a5a162e65b99898ec12fbe8e8ee59664e56ead405 WatchSource:0}: Error finding container ed2c90ae4cdb165483962a7a5a162e65b99898ec12fbe8e8ee59664e56ead405: Status 404 returned error can't find the container with id ed2c90ae4cdb165483962a7a5a162e65b99898ec12fbe8e8ee59664e56ead405
Sep 19 09:35:27.813043 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.812948    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw" event={"ID":"95df44b9-143b-4f44-8510-85739e72fb74","Type":"ContainerStarted","Data":"0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006"}
Sep 19 09:35:27.813043 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.812992    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw" event={"ID":"95df44b9-143b-4f44-8510-85739e72fb74","Type":"ContainerStarted","Data":"ed2c90ae4cdb165483962a7a5a162e65b99898ec12fbe8e8ee59664e56ead405"}
Sep 19 09:35:27.831481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:27.831433    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw" podStartSLOduration=1.8314153370000001 podStartE2EDuration="1.831415337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:35:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:35:27.830562971 +0000 UTC m=+1741.517368271" watchObservedRunningTime="2026-09-19 09:35:27.831415337 +0000 UTC m=+1741.518220636"
Sep 19 09:35:28.046325 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:28.046291    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:28.826409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:28.826372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:29.583063 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:29.583021    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:30.311347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:30.311318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:31.225143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:31.225109    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:37.184699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:37.184665    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:37.184699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:37.184710    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:37.197457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:37.197426    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:37.878623 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:37.878595    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:35:40.863254 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:40.863218    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:40.936752 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:40.936718    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:41.001717 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:41.001680    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:41.065298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:41.065258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:41.132422 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:41.132391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:41.517398 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:41.517310    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:42.237602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:42.237567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:43.009601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:43.009569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:43.804321 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:43.804281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:44.346726 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:44.346684    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:44.494028 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:44.493988    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:45.167231 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:45.167187    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:35:48.070702 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:48.070666    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:35:49.327153 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:49.327118    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"]
Sep 19 09:35:49.420692 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:35:49.420652    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:04.346592 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.346521    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82" podUID="06d60781-d5e8-4acf-9e0a-967978d5b306" containerName="main" containerID="cri-o://d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e" gracePeriod=45
Sep 19 09:36:04.597570 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.597502    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:36:04.729965 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.729927    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-dshm\") pod \"06d60781-d5e8-4acf-9e0a-967978d5b306\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") "
Sep 19 09:36:04.730195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.729989    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-model-cache\") pod \"06d60781-d5e8-4acf-9e0a-967978d5b306\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") "
Sep 19 09:36:04.730195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730010    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwjjb\" (UniqueName: \"kubernetes.io/projected/06d60781-d5e8-4acf-9e0a-967978d5b306-kube-api-access-gwjjb\") pod \"06d60781-d5e8-4acf-9e0a-967978d5b306\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") "
Sep 19 09:36:04.730195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730032    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-home\") pod \"06d60781-d5e8-4acf-9e0a-967978d5b306\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") "
Sep 19 09:36:04.730195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730070    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d60781-d5e8-4acf-9e0a-967978d5b306-tls-certs\") pod \"06d60781-d5e8-4acf-9e0a-967978d5b306\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") "
Sep 19 09:36:04.730195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730174    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-tmp-dir\") pod \"06d60781-d5e8-4acf-9e0a-967978d5b306\" (UID: \"06d60781-d5e8-4acf-9e0a-967978d5b306\") "
Sep 19 09:36:04.730482 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730318    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-model-cache" (OuterVolumeSpecName: "model-cache") pod "06d60781-d5e8-4acf-9e0a-967978d5b306" (UID: "06d60781-d5e8-4acf-9e0a-967978d5b306"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:04.730482 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730359    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-home" (OuterVolumeSpecName: "home") pod "06d60781-d5e8-4acf-9e0a-967978d5b306" (UID: "06d60781-d5e8-4acf-9e0a-967978d5b306"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:04.730597 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730487    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06d60781-d5e8-4acf-9e0a-967978d5b306" (UID: "06d60781-d5e8-4acf-9e0a-967978d5b306"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:04.730654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730601    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:04.730654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730613    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:04.730654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.730621    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:04.732557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.732529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d60781-d5e8-4acf-9e0a-967978d5b306-kube-api-access-gwjjb" (OuterVolumeSpecName: "kube-api-access-gwjjb") pod "06d60781-d5e8-4acf-9e0a-967978d5b306" (UID: "06d60781-d5e8-4acf-9e0a-967978d5b306"). InnerVolumeSpecName "kube-api-access-gwjjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:36:04.732557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.732546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06d60781-d5e8-4acf-9e0a-967978d5b306-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06d60781-d5e8-4acf-9e0a-967978d5b306" (UID: "06d60781-d5e8-4acf-9e0a-967978d5b306"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:36:04.732557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.732552    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-dshm" (OuterVolumeSpecName: "dshm") pod "06d60781-d5e8-4acf-9e0a-967978d5b306" (UID: "06d60781-d5e8-4acf-9e0a-967978d5b306"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:04.831614 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.831574    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwjjb\" (UniqueName: \"kubernetes.io/projected/06d60781-d5e8-4acf-9e0a-967978d5b306-kube-api-access-gwjjb\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:04.831614 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.831605    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d60781-d5e8-4acf-9e0a-967978d5b306-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:04.831614 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:04.831617    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d60781-d5e8-4acf-9e0a-967978d5b306-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:05.010992 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.010960    2581 generic.go:356] "Generic (PLEG): container finished" podID="06d60781-d5e8-4acf-9e0a-967978d5b306" containerID="d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e" exitCode=0
Sep 19 09:36:05.010992 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.010993    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"
Sep 19 09:36:05.011280 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.011030    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82" event={"ID":"06d60781-d5e8-4acf-9e0a-967978d5b306","Type":"ContainerDied","Data":"d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e"}
Sep 19 09:36:05.011280 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.011074    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82" event={"ID":"06d60781-d5e8-4acf-9e0a-967978d5b306","Type":"ContainerDied","Data":"5343deaf9ac80865109a5d2f34492d2a324ebd04c52180c604675a35e0a6eb5a"}
Sep 19 09:36:05.011280 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.011090    2581 scope.go:117] "RemoveContainer" containerID="d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e"
Sep 19 09:36:05.019867 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.019849    2581 scope.go:117] "RemoveContainer" containerID="d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e"
Sep 19 09:36:05.020193 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:36:05.020174    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e\": container with ID starting with d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e not found: ID does not exist" containerID="d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e"
Sep 19 09:36:05.020258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.020201    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e"} err="failed to get container status \"d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e\": rpc error: code = NotFound desc = could not find container \"d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e\": container with ID starting with d3b35e1ef3d2ecdf9391eda7b2e7c96a4404752f0747a9feb3a234b0daf9af0e not found: ID does not exist"
Sep 19 09:36:05.027954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.027931    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"]
Sep 19 09:36:05.034289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:05.034266    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82"]
Sep 19 09:36:06.830917 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:06.830881    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06d60781-d5e8-4acf-9e0a-967978d5b306" path="/var/lib/kubelet/pods/06d60781-d5e8-4acf-9e0a-967978d5b306/volumes"
Sep 19 09:36:09.534557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:09.534502    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-e46994f4/e2e-lifecycle-model-e46994f4-kserve-5b9bcd56d7-6zk82" podUID="06d60781-d5e8-4acf-9e0a-967978d5b306" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 19 09:36:12.418936 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:12.418900    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:36:20.406601 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:20.406567    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:26.056221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:26.056185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:26.505258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:26.505228    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:27.063188 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:27.063155    2581 scope.go:117] "RemoveContainer" containerID="eab5c15694c7a02ccb30bf7c3a6d7f45e2a54f80ea7a9dba236dbb20cf4014be"
Sep 19 09:36:27.073714 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:27.073689    2581 scope.go:117] "RemoveContainer" containerID="dbebf3a7e5c43edf3c4873f301fb26fb88e7b619a8bdec83bb7a024e70951c19"
Sep 19 09:36:28.571324 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.571290    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"]
Sep 19 09:36:28.571713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.571667    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d60781-d5e8-4acf-9e0a-967978d5b306" containerName="main"
Sep 19 09:36:28.571713 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.571680    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d60781-d5e8-4acf-9e0a-967978d5b306" containerName="main"
Sep 19 09:36:28.571785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.571747    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d60781-d5e8-4acf-9e0a-967978d5b306" containerName="main"
Sep 19 09:36:28.574669 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.574651    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.578258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.578231    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"kube-root-ca.crt\""
Sep 19 09:36:28.578495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.578477    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"openshift-service-ca.crt\""
Sep 19 09:36:28.578585 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.578495    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"auth-test-model-1afa13-kserve-self-signed-certs\""
Sep 19 09:36:28.578585 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.578544    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"default-dockercfg-g8624\""
Sep 19 09:36:28.583888 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.583866    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"]
Sep 19 09:36:28.656567 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.656537    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-dshm\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.656567 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.656574    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-model-cache\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.656832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.656607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-tmp-dir\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.656832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.656652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-home\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.656832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.656748    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7vjb\" (UniqueName: \"kubernetes.io/projected/16369e5b-22ee-4368-b364-37820c889fcc-kube-api-access-p7vjb\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.656943 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.656863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16369e5b-22ee-4368-b364-37820c889fcc-tls-certs\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.676840 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.676805    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:28.757394 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757353    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7vjb\" (UniqueName: \"kubernetes.io/projected/16369e5b-22ee-4368-b364-37820c889fcc-kube-api-access-p7vjb\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.757580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16369e5b-22ee-4368-b364-37820c889fcc-tls-certs\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.757580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-dshm\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.757580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-model-cache\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.757580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757505    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-tmp-dir\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.757580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-home\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.757942 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757920    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-tmp-dir\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.758018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.757967    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-model-cache\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.758018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.758006    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-home\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.759922 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.759894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-dshm\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.760341 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.760323    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16369e5b-22ee-4368-b364-37820c889fcc-tls-certs\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.766603 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.766576    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7vjb\" (UniqueName: \"kubernetes.io/projected/16369e5b-22ee-4368-b364-37820c889fcc-kube-api-access-p7vjb\") pod \"auth-test-model-1afa13-kserve-7948c89dd7-66hh5\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:28.884958 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:28.884921    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:29.164729 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:29.164538    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"]
Sep 19 09:36:29.167433 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:36:29.167406    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16369e5b_22ee_4368_b364_37820c889fcc.slice/crio-f7c8c4fde43ce723075a7f203f81618e51a07008271cc761f56a22cc21e91e2e WatchSource:0}: Error finding container f7c8c4fde43ce723075a7f203f81618e51a07008271cc761f56a22cc21e91e2e: Status 404 returned error can't find the container with id f7c8c4fde43ce723075a7f203f81618e51a07008271cc761f56a22cc21e91e2e
Sep 19 09:36:30.147512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:30.147475    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5" event={"ID":"16369e5b-22ee-4368-b364-37820c889fcc","Type":"ContainerStarted","Data":"0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834"}
Sep 19 09:36:30.147512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:30.147516    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5" event={"ID":"16369e5b-22ee-4368-b364-37820c889fcc","Type":"ContainerStarted","Data":"f7c8c4fde43ce723075a7f203f81618e51a07008271cc761f56a22cc21e91e2e"}
Sep 19 09:36:30.166133 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:30.166067    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5" podStartSLOduration=2.166050526 podStartE2EDuration="2.166050526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:36:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:36:30.164703091 +0000 UTC m=+1803.851508392" watchObservedRunningTime="2026-09-19 09:36:30.166050526 +0000 UTC m=+1803.852855850"
Sep 19 09:36:32.362966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:32.362926    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:32.553559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:32.553528    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:32.600032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:32.600002    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"]
Sep 19 09:36:32.600312 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:32.600285    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" containerID="cri-o://a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078" gracePeriod=30
Sep 19 09:36:32.628808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:32.628769    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:33.861653 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:33.861613    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:33.956322 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:33.956286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:34.056590 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:34.056552    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:34.644935 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:34.644890    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:34.741923 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:34.741891    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:34.843024 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:34.842983    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:36.098832 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:36.098795    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:36.572276 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:36.572241    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:36:36.650398 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:36.650361    2581 patch_prober.go:28] interesting pod/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 19 09:36:36.650567 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:36.650490    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 09:36:37.857227 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.857198    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:36:37.948946 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.948910    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-podinfo\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.948970    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-envoy\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949009    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istiod-ca-cert\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949024    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-socket\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949090    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-token\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949149    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zppxs\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-kube-api-access-zppxs\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949183    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-data\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949240    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-credential-socket\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949269    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-certs\") pod \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\" (UID: \"00bb2ea4-256a-40c7-be55-7bda3cdf81a7\") "
Sep 19 09:36:37.949526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949400    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:37.949672 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949532    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:36:37.949672 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949629    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:37.949947 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949705    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-credential-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:37.949947 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949725    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istiod-ca-cert\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:37.949947 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949723    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:37.949947 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.949739    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:37.950849 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.950822    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:37.951619 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.951592    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:36:37.951701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.951637    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 09:36:37.951803 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.951782    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:36:37.951845 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:37.951802    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-kube-api-access-zppxs" (OuterVolumeSpecName: "kube-api-access-zppxs") pod "00bb2ea4-256a-40c7-be55-7bda3cdf81a7" (UID: "00bb2ea4-256a-40c7-be55-7bda3cdf81a7"). InnerVolumeSpecName "kube-api-access-zppxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:36:38.050805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.050768    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zppxs\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-kube-api-access-zppxs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:38.050805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.050797    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-data\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:38.050805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.050807    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-workload-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:38.050805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.050814    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-podinfo\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:38.051080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.050822    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-envoy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:38.051080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.050830    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00bb2ea4-256a-40c7-be55-7bda3cdf81a7-istio-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:36:38.194500 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.194457    2581 generic.go:356] "Generic (PLEG): container finished" podID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerID="a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078" exitCode=0
Sep 19 09:36:38.194680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.194521    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"
Sep 19 09:36:38.194680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.194528    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" event={"ID":"00bb2ea4-256a-40c7-be55-7bda3cdf81a7","Type":"ContainerDied","Data":"a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078"}
Sep 19 09:36:38.194680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.194612    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm" event={"ID":"00bb2ea4-256a-40c7-be55-7bda3cdf81a7","Type":"ContainerDied","Data":"f71761300ce65818a79d1e14c059ba2c399f9cdbcf812b9b668e28fa43bdcac8"}
Sep 19 09:36:38.194680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.194640    2581 scope.go:117] "RemoveContainer" containerID="a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078"
Sep 19 09:36:38.205032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.205013    2581 scope.go:117] "RemoveContainer" containerID="3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84"
Sep 19 09:36:38.216007 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.215972    2581 scope.go:117] "RemoveContainer" containerID="a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078"
Sep 19 09:36:38.217408 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:36:38.217374    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078\": container with ID starting with a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078 not found: ID does not exist" containerID="a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078"
Sep 19 09:36:38.217534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.217423    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078"} err="failed to get container status \"a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078\": rpc error: code = NotFound desc = could not find container \"a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078\": container with ID starting with a26a6dcf2f65a0479e4f4c23977e5a6e6b1a6628cfa5489c50db62e768f79078 not found: ID does not exist"
Sep 19 09:36:38.217534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.217458    2581 scope.go:117] "RemoveContainer" containerID="3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84"
Sep 19 09:36:38.217828 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:36:38.217807    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84\": container with ID starting with 3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84 not found: ID does not exist" containerID="3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84"
Sep 19 09:36:38.217921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.217832    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84"} err="failed to get container status \"3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84\": rpc error: code = NotFound desc = could not find container \"3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84\": container with ID starting with 3899fa3dbf080a2f9cfa28665fc37dda119ef934236cb7ab8b088eb634a38c84 not found: ID does not exist"
Sep 19 09:36:38.219287 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.219268    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"]
Sep 19 09:36:38.223544 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.223518    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e46994f4-openshift-default-fbcb4768-bzgzm"]
Sep 19 09:36:38.831073 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.831040    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" path="/var/lib/kubelet/pods/00bb2ea4-256a-40c7-be55-7bda3cdf81a7/volumes"
Sep 19 09:36:38.885600 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.885569    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:38.886004 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.885616    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:38.899775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:38.899746    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:39.211775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:39.211749    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:36:42.076649 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:42.076606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:42.149435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:42.149401    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:42.738395 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:42.738360    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:48.558996 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:48.558959    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:48.668851 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:48.668818    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:48.757427 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:48.757400    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:49.794905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:49.794864    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:54.473819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.473786    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"]
Sep 19 09:36:54.474212 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.474158    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy"
Sep 19 09:36:54.474212 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.474171    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy"
Sep 19 09:36:54.474289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.474237    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy"
Sep 19 09:36:54.474289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.474245    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy"
Sep 19 09:36:54.478761 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.478739    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.481710 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.481690    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-14e3f6\"/\"embedding-w0-14e3f6-kserve-self-signed-certs\""
Sep 19 09:36:54.488566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.488540    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"]
Sep 19 09:36:54.502215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.502185    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:54.610950 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.610913    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc9m9\" (UniqueName: \"kubernetes.io/projected/ce478e8e-17a6-4c52-8106-283ebe4dd74d-kube-api-access-tc9m9\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.611172 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.610966    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-home\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.611172 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.610985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-model-cache\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.611172 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.611068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-dshm\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.611328 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.611179    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tmp-dir\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.611328 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.611253    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tls-certs\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.712559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.712513    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tls-certs\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.712768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.712572    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc9m9\" (UniqueName: \"kubernetes.io/projected/ce478e8e-17a6-4c52-8106-283ebe4dd74d-kube-api-access-tc9m9\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.712768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.712621    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-home\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.712768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.712642    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-model-cache\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.712768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.712694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-dshm\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.713004 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.712849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tmp-dir\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.713062 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.713044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-home\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.713159 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.713140    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-model-cache\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.713210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.713174    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tmp-dir\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.715209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.715186    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-dshm\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.715390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.715371    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tls-certs\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.720838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.720814    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc9m9\" (UniqueName: \"kubernetes.io/projected/ce478e8e-17a6-4c52-8106-283ebe4dd74d-kube-api-access-tc9m9\") pod \"embedding-w0-14e3f6-kserve-564cb766c5-xf9wp\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") " pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:54.789134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:54.789023    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:36:55.065066 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:55.065039    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"]
Sep 19 09:36:55.067181 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:36:55.067151    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce478e8e_17a6_4c52_8106_283ebe4dd74d.slice/crio-1d5d5ff76b9d3d5f8e7f8665009b6aca080343cd91aa530bb1c39f3a98ad6f94 WatchSource:0}: Error finding container 1d5d5ff76b9d3d5f8e7f8665009b6aca080343cd91aa530bb1c39f3a98ad6f94: Status 404 returned error can't find the container with id 1d5d5ff76b9d3d5f8e7f8665009b6aca080343cd91aa530bb1c39f3a98ad6f94
Sep 19 09:36:55.282118 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:55.282068    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp" event={"ID":"ce478e8e-17a6-4c52-8106-283ebe4dd74d","Type":"ContainerStarted","Data":"c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5"}
Sep 19 09:36:55.282298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:55.282131    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp" event={"ID":"ce478e8e-17a6-4c52-8106-283ebe4dd74d","Type":"ContainerStarted","Data":"1d5d5ff76b9d3d5f8e7f8665009b6aca080343cd91aa530bb1c39f3a98ad6f94"}
Sep 19 09:36:55.300516 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:55.300463    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp" podStartSLOduration=1.300446027 podStartE2EDuration="1.300446027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:36:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:36:55.299055512 +0000 UTC m=+1828.985860811" watchObservedRunningTime="2026-09-19 09:36:55.300446027 +0000 UTC m=+1828.987251325"
Sep 19 09:36:59.236511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:59.236467    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:36:59.464452 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:36:59.464421    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:00.384319 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:00.384280    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:37:04.790189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:04.790145    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:37:04.790189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:04.790202    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:37:04.802648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:04.802621    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:37:05.345326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:05.345289    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:37:14.402235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:14.402200    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:20.362185 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:20.362147    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:24.572608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:24.572563    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:37:26.260498 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:26.260456    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:26.324906 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:26.324875    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:26.610663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:26.610550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:26.972011 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:26.971976    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:27.142300 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:27.142260    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:29.382857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:29.382812    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:30.030039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:30.030003    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:30.123042 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:30.123013    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:30.224228 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:30.224195    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:30.780324 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:30.780288    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:31.038633 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:31.038553    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:31.987788 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:31.987746    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:32.657789 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:32.657748    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:33.408837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:33.408798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:34.220448 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:34.220415    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:34.317269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:34.317229    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:34.416574 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:34.416535    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:34.701267 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:34.701227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:34.793706 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:34.793667    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:49.883513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:49.883425    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:37:56.617752 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:56.617698    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:57.516521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:57.516481    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:58.218481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:58.218437    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:59.014971 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:59.014934    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:37:59.817748 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:37:59.817708    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:00.092429 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:00.092332    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:00.543875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:00.543844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:00.662082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:00.662047    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:00.885918 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:00.885888    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:01.029307 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:01.029277    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:01.559684 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:01.559638    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:03.711963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:03.711927    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:04.295860 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:04.295822    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:04.439462 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:04.439427    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:04.818310 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:04.818264    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:05.813515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:05.813483    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:06.657519 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:06.657469    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:07.546805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:07.546767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:08.385786 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:08.385746    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:09.185026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.184998    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:09.874545 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.874514    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"]
Sep 19 09:38:09.874919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.874858    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy"
Sep 19 09:38:09.874919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.874869    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="00bb2ea4-256a-40c7-be55-7bda3cdf81a7" containerName="istio-proxy"
Sep 19 09:38:09.877872 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.877850    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:09.880271 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.880246    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0304ec-serving-cert\""
Sep 19 09:38:09.889921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.889896    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"]
Sep 19 09:38:09.908515 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.908484    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:09.969221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:09.969183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:10.025413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.025380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdmvg\" (UniqueName: \"kubernetes.io/projected/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-kube-api-access-sdmvg\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.025596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.025429    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-metrics-certs\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.025596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.025494    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-openshift-service-ca\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.025596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.025539    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-maas-api-tls\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.126082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.126013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-metrics-certs\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.126082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.126056    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-openshift-service-ca\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.126082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.126077    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-maas-api-tls\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.126416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.126147    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdmvg\" (UniqueName: \"kubernetes.io/projected/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-kube-api-access-sdmvg\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.126754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.126728    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-openshift-service-ca\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.128779 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.128757    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-maas-api-tls\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.128874 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.128801    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-metrics-certs\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.137370 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.137346    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdmvg\" (UniqueName: \"kubernetes.io/projected/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-kube-api-access-sdmvg\") pod \"maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.164711 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.164675    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:10.188482 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.188446    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:10.426954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.426860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:10.465911 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.465788    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"]
Sep 19 09:38:10.468204 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:38:10.468176    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff5ec2c_0445_4e74_ac9a_68ac3f0967ef.slice/crio-aa67d64cd183efafe4ab37dddb89d30e8db8fc4228927118aa74885a316eb4dc WatchSource:0}: Error finding container aa67d64cd183efafe4ab37dddb89d30e8db8fc4228927118aa74885a316eb4dc: Status 404 returned error can't find the container with id aa67d64cd183efafe4ab37dddb89d30e8db8fc4228927118aa74885a316eb4dc
Sep 19 09:38:10.469644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.469626    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 09:38:10.523569 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.523537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:10.570819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.570781    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"]
Sep 19 09:38:10.575366 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.575342    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:10.577825 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.577804    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0304ec\""
Sep 19 09:38:10.577960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.577943    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0304ec-dockercfg-54w9c\""
Sep 19 09:38:10.586459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.586428    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"]
Sep 19 09:38:10.672672 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.672635    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8" event={"ID":"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef","Type":"ContainerStarted","Data":"aa67d64cd183efafe4ab37dddb89d30e8db8fc4228927118aa74885a316eb4dc"}
Sep 19 09:38:10.732789 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.732706    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03df881e-d96f-42f1-ad66-0cb1f5add5a7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:10.732789 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.732767    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcrc6\" (UniqueName: \"kubernetes.io/projected/03df881e-d96f-42f1-ad66-0cb1f5add5a7-kube-api-access-tcrc6\") pod \"payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:10.834109 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.834057    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03df881e-d96f-42f1-ad66-0cb1f5add5a7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:10.834209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.834179    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcrc6\" (UniqueName: \"kubernetes.io/projected/03df881e-d96f-42f1-ad66-0cb1f5add5a7-kube-api-access-tcrc6\") pod \"payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:10.834809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.834784    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03df881e-d96f-42f1-ad66-0cb1f5add5a7-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:10.842804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.842782    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcrc6\" (UniqueName: \"kubernetes.io/projected/03df881e-d96f-42f1-ad66-0cb1f5add5a7-kube-api-access-tcrc6\") pod \"payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") " pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:10.887321 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:10.887290    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:11.167821 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.167794    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"]
Sep 19 09:38:11.169820 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:38:11.169795    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03df881e_d96f_42f1_ad66_0cb1f5add5a7.slice/crio-ac084e4fb83156c680ba11408db93c0538d603accaa382f3cf52100f7daa711f WatchSource:0}: Error finding container ac084e4fb83156c680ba11408db93c0538d603accaa382f3cf52100f7daa711f: Status 404 returned error can't find the container with id ac084e4fb83156c680ba11408db93c0538d603accaa382f3cf52100f7daa711f
Sep 19 09:38:11.681154 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.681091    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8" event={"ID":"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef","Type":"ContainerStarted","Data":"a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7"}
Sep 19 09:38:11.681354 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.681185    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:11.682777 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.682753    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs" event={"ID":"03df881e-d96f-42f1-ad66-0cb1f5add5a7","Type":"ContainerStarted","Data":"fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c"}
Sep 19 09:38:11.682878 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.682797    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs" event={"ID":"03df881e-d96f-42f1-ad66-0cb1f5add5a7","Type":"ContainerStarted","Data":"ac084e4fb83156c680ba11408db93c0538d603accaa382f3cf52100f7daa711f"}
Sep 19 09:38:11.682878 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.682825    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:11.700937 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.700885    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8" podStartSLOduration=2.350680323 podStartE2EDuration="2.700869766s" totalImagesPullingTime="350.189443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:38:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:38:11.698839995 +0000 UTC m=+1905.385645301" watchObservedRunningTime="2026-09-19 09:38:11.700869766 +0000 UTC m=+1905.387675043"
Sep 19 09:38:11.717977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:11.717918    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs" podStartSLOduration=1.717896251 podStartE2EDuration="1.717896251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:38:11.715133952 +0000 UTC m=+1905.401939251" watchObservedRunningTime="2026-09-19 09:38:11.717896251 +0000 UTC m=+1905.404701552"
Sep 19 09:38:12.025263 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:12.025169    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:12.369772 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:12.369682    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:13.183822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:13.183788    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:13.922300 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:13.922252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:14.607121 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:14.607061    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:15.407667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:15.407633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:16.104844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:16.104811    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:16.808975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:16.808940    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:17.019033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.018997    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"]
Sep 19 09:38:17.033332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.033295    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"]
Sep 19 09:38:17.057582 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.057553    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"]
Sep 19 09:38:17.089003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.088909    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"]
Sep 19 09:38:17.119787 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.119753    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"]
Sep 19 09:38:17.259171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.259094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:17.694793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.694761    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:38:17.811430 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:17.811393    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:18.641577 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:18.641521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:21.381454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:21.381417    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:38:22.691980 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:22.691949    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:38:27.846465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:27.846432    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"]
Sep 19 09:38:27.852844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:27.852806    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:27.853004 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:27.852870    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-14e3f6-29830170-rf2gt"]
Sep 19 09:38:27.926865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:27.926831    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:28.024833 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:28.024796    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:28.114005 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:28.113901    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:28.179732 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:28.179681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:28.453581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:28.453550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:28.612572 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:28.612523    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:28.831261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:28.831177    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301" path="/var/lib/kubelet/pods/f0bc30f5-d5dd-493f-a2ae-f71a0e1b0301/volumes"
Sep 19 09:38:29.532317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:29.532262    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:30.468660 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:30.468622    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:31.287681 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:31.287646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:32.039516 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.039448    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67" podUID="340bfca3-6480-43e8-9dcb-2eb7765fb1dc" containerName="main" containerID="cri-o://49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd" gracePeriod=45
Sep 19 09:38:32.052716 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.052648    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf" podUID="844f2cc3-0f3a-4c0b-9015-149a1f935650" containerName="main" containerID="cri-o://9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba" gracePeriod=45
Sep 19 09:38:32.077364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.077304    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp" podUID="ce478e8e-17a6-4c52-8106-283ebe4dd74d" containerName="main" containerID="cri-o://c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5" gracePeriod=45
Sep 19 09:38:32.108694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.108659    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg" podUID="63cf5320-3ebf-4c2a-a3ce-b27e277d407c" containerName="main" containerID="cri-o://204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712" gracePeriod=45
Sep 19 09:38:32.116983 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.116956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:32.139296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.139233    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm" podUID="36aa4868-42ef-4a86-ba60-bd1d18c19d06" containerName="main" containerID="cri-o://442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a" gracePeriod=45
Sep 19 09:38:32.367501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.367477    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:38:32.428387 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.428358    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:38:32.442058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442025    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-home\") pod \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") "
Sep 19 09:38:32.442255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442077    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-dshm\") pod \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") "
Sep 19 09:38:32.442255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442150    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpwcn\" (UniqueName: \"kubernetes.io/projected/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-kube-api-access-kpwcn\") pod \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") "
Sep 19 09:38:32.442255 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442191    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tmp-dir\") pod \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") "
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442263    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/844f2cc3-0f3a-4c0b-9015-149a1f935650-tls-certs\") pod \"844f2cc3-0f3a-4c0b-9015-149a1f935650\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") "
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442291    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tls-certs\") pod \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") "
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442319    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-model-cache\") pod \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\" (UID: \"340bfca3-6480-43e8-9dcb-2eb7765fb1dc\") "
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442325    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-home" (OuterVolumeSpecName: "home") pod "340bfca3-6480-43e8-9dcb-2eb7765fb1dc" (UID: "340bfca3-6480-43e8-9dcb-2eb7765fb1dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442355    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-model-cache\") pod \"844f2cc3-0f3a-4c0b-9015-149a1f935650\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") "
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442379    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-home\") pod \"844f2cc3-0f3a-4c0b-9015-149a1f935650\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") "
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442421    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-dshm\") pod \"844f2cc3-0f3a-4c0b-9015-149a1f935650\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") "
Sep 19 09:38:32.442471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442470    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-tmp-dir\") pod \"844f2cc3-0f3a-4c0b-9015-149a1f935650\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") "
Sep 19 09:38:32.442862 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442502    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzthr\" (UniqueName: \"kubernetes.io/projected/844f2cc3-0f3a-4c0b-9015-149a1f935650-kube-api-access-gzthr\") pod \"844f2cc3-0f3a-4c0b-9015-149a1f935650\" (UID: \"844f2cc3-0f3a-4c0b-9015-149a1f935650\") "
Sep 19 09:38:32.442862 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.442802    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.443448 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.443284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-model-cache" (OuterVolumeSpecName: "model-cache") pod "844f2cc3-0f3a-4c0b-9015-149a1f935650" (UID: "844f2cc3-0f3a-4c0b-9015-149a1f935650"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.443448 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.443334    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "340bfca3-6480-43e8-9dcb-2eb7765fb1dc" (UID: "340bfca3-6480-43e8-9dcb-2eb7765fb1dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.443641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.443529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-home" (OuterVolumeSpecName: "home") pod "844f2cc3-0f3a-4c0b-9015-149a1f935650" (UID: "844f2cc3-0f3a-4c0b-9015-149a1f935650"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.443731 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.443695    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "844f2cc3-0f3a-4c0b-9015-149a1f935650" (UID: "844f2cc3-0f3a-4c0b-9015-149a1f935650"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.443837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.443742    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "340bfca3-6480-43e8-9dcb-2eb7765fb1dc" (UID: "340bfca3-6480-43e8-9dcb-2eb7765fb1dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.449557 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.449518    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/844f2cc3-0f3a-4c0b-9015-149a1f935650-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "844f2cc3-0f3a-4c0b-9015-149a1f935650" (UID: "844f2cc3-0f3a-4c0b-9015-149a1f935650"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:38:32.449895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.449867    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-kube-api-access-kpwcn" (OuterVolumeSpecName: "kube-api-access-kpwcn") pod "340bfca3-6480-43e8-9dcb-2eb7765fb1dc" (UID: "340bfca3-6480-43e8-9dcb-2eb7765fb1dc"). InnerVolumeSpecName "kube-api-access-kpwcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:38:32.450513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.450475    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-dshm" (OuterVolumeSpecName: "dshm") pod "340bfca3-6480-43e8-9dcb-2eb7765fb1dc" (UID: "340bfca3-6480-43e8-9dcb-2eb7765fb1dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.450618 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.450511    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "340bfca3-6480-43e8-9dcb-2eb7765fb1dc" (UID: "340bfca3-6480-43e8-9dcb-2eb7765fb1dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:38:32.450897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.450864    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-dshm" (OuterVolumeSpecName: "dshm") pod "844f2cc3-0f3a-4c0b-9015-149a1f935650" (UID: "844f2cc3-0f3a-4c0b-9015-149a1f935650"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.451401 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.451352    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/844f2cc3-0f3a-4c0b-9015-149a1f935650-kube-api-access-gzthr" (OuterVolumeSpecName: "kube-api-access-gzthr") pod "844f2cc3-0f3a-4c0b-9015-149a1f935650" (UID: "844f2cc3-0f3a-4c0b-9015-149a1f935650"). InnerVolumeSpecName "kube-api-access-gzthr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:38:32.467552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.467528    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:38:32.471081 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.471052    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:38:32.475583 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.475559    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:38:32.544012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.543983    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-model-cache\") pod \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") "
Sep 19 09:38:32.544012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544015    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-home\") pod \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544031    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-dshm\") pod \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544060    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-home\") pod \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544093    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-model-cache\") pod \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544142    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tls-certs\") pod \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544165    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tmp-dir\") pod \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544197    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrsrg\" (UniqueName: \"kubernetes.io/projected/36aa4868-42ef-4a86-ba60-bd1d18c19d06-kube-api-access-mrsrg\") pod \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544221    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tmp-dir\") pod \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") "
Sep 19 09:38:32.544272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544259    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tls-certs\") pod \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544307    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tmp-dir\") pod \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544319    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-home" (OuterVolumeSpecName: "home") pod "63cf5320-3ebf-4c2a-a3ce-b27e277d407c" (UID: "63cf5320-3ebf-4c2a-a3ce-b27e277d407c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544333    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-model-cache" (OuterVolumeSpecName: "model-cache") pod "63cf5320-3ebf-4c2a-a3ce-b27e277d407c" (UID: "63cf5320-3ebf-4c2a-a3ce-b27e277d407c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544345    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tls-certs\") pod \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544358    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-home" (OuterVolumeSpecName: "home") pod "ce478e8e-17a6-4c52-8106-283ebe4dd74d" (UID: "ce478e8e-17a6-4c52-8106-283ebe4dd74d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544407    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc9m9\" (UniqueName: \"kubernetes.io/projected/ce478e8e-17a6-4c52-8106-283ebe4dd74d-kube-api-access-tc9m9\") pod \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\" (UID: \"ce478e8e-17a6-4c52-8106-283ebe4dd74d\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544486    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-dshm\") pod \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544521    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg7zk\" (UniqueName: \"kubernetes.io/projected/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-kube-api-access-dg7zk\") pod \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544564    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-dshm\") pod \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\" (UID: \"63cf5320-3ebf-4c2a-a3ce-b27e277d407c\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544589    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-model-cache\") pod \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") "
Sep 19 09:38:32.544699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544620    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-home\") pod \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\" (UID: \"36aa4868-42ef-4a86-ba60-bd1d18c19d06\") "
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.544943    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce478e8e-17a6-4c52-8106-283ebe4dd74d" (UID: "ce478e8e-17a6-4c52-8106-283ebe4dd74d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545262    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/844f2cc3-0f3a-4c0b-9015-149a1f935650-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545283    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545297    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545309    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545322    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545284    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-home" (OuterVolumeSpecName: "home") pod "36aa4868-42ef-4a86-ba60-bd1d18c19d06" (UID: "36aa4868-42ef-4a86-ba60-bd1d18c19d06"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545333    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545365    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/844f2cc3-0f3a-4c0b-9015-149a1f935650-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545384    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzthr\" (UniqueName: \"kubernetes.io/projected/844f2cc3-0f3a-4c0b-9015-149a1f935650-kube-api-access-gzthr\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545399    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545414    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpwcn\" (UniqueName: \"kubernetes.io/projected/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-kube-api-access-kpwcn\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545429    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.545437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545443    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.546194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545455    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340bfca3-6480-43e8-9dcb-2eb7765fb1dc-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.546194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545468    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.546194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545480    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.546194 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.545616    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36aa4868-42ef-4a86-ba60-bd1d18c19d06" (UID: "36aa4868-42ef-4a86-ba60-bd1d18c19d06"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.546415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.546191    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-model-cache" (OuterVolumeSpecName: "model-cache") pod "36aa4868-42ef-4a86-ba60-bd1d18c19d06" (UID: "36aa4868-42ef-4a86-ba60-bd1d18c19d06"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.546415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.546291    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce478e8e-17a6-4c52-8106-283ebe4dd74d" (UID: "ce478e8e-17a6-4c52-8106-283ebe4dd74d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.546415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.546359    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63cf5320-3ebf-4c2a-a3ce-b27e277d407c" (UID: "63cf5320-3ebf-4c2a-a3ce-b27e277d407c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.548540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.548499    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-dshm" (OuterVolumeSpecName: "dshm") pod "63cf5320-3ebf-4c2a-a3ce-b27e277d407c" (UID: "63cf5320-3ebf-4c2a-a3ce-b27e277d407c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.548675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.548544    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-kube-api-access-dg7zk" (OuterVolumeSpecName: "kube-api-access-dg7zk") pod "63cf5320-3ebf-4c2a-a3ce-b27e277d407c" (UID: "63cf5320-3ebf-4c2a-a3ce-b27e277d407c"). InnerVolumeSpecName "kube-api-access-dg7zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:38:32.548675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.548629    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63cf5320-3ebf-4c2a-a3ce-b27e277d407c" (UID: "63cf5320-3ebf-4c2a-a3ce-b27e277d407c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:38:32.549077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.548775    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36aa4868-42ef-4a86-ba60-bd1d18c19d06-kube-api-access-mrsrg" (OuterVolumeSpecName: "kube-api-access-mrsrg") pod "36aa4868-42ef-4a86-ba60-bd1d18c19d06" (UID: "36aa4868-42ef-4a86-ba60-bd1d18c19d06"). InnerVolumeSpecName "kube-api-access-mrsrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:38:32.549077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.549076    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36aa4868-42ef-4a86-ba60-bd1d18c19d06" (UID: "36aa4868-42ef-4a86-ba60-bd1d18c19d06"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:38:32.549077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.549076    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-dshm" (OuterVolumeSpecName: "dshm") pod "ce478e8e-17a6-4c52-8106-283ebe4dd74d" (UID: "ce478e8e-17a6-4c52-8106-283ebe4dd74d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.549393 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.549364    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-dshm" (OuterVolumeSpecName: "dshm") pod "36aa4868-42ef-4a86-ba60-bd1d18c19d06" (UID: "36aa4868-42ef-4a86-ba60-bd1d18c19d06"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:38:32.549932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.549907    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce478e8e-17a6-4c52-8106-283ebe4dd74d" (UID: "ce478e8e-17a6-4c52-8106-283ebe4dd74d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:38:32.550587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.550566    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce478e8e-17a6-4c52-8106-283ebe4dd74d-kube-api-access-tc9m9" (OuterVolumeSpecName: "kube-api-access-tc9m9") pod "ce478e8e-17a6-4c52-8106-283ebe4dd74d" (UID: "ce478e8e-17a6-4c52-8106-283ebe4dd74d"). InnerVolumeSpecName "kube-api-access-tc9m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:38:32.646474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646441    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646471    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce478e8e-17a6-4c52-8106-283ebe4dd74d-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646474 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646480    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc9m9\" (UniqueName: \"kubernetes.io/projected/ce478e8e-17a6-4c52-8106-283ebe4dd74d-kube-api-access-tc9m9\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646489    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646498    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg7zk\" (UniqueName: \"kubernetes.io/projected/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-kube-api-access-dg7zk\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646507    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646516    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646524    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646531    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce478e8e-17a6-4c52-8106-283ebe4dd74d-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646538    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646545    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36aa4868-42ef-4a86-ba60-bd1d18c19d06-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646552    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrsrg\" (UniqueName: \"kubernetes.io/projected/36aa4868-42ef-4a86-ba60-bd1d18c19d06-kube-api-access-mrsrg\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646559    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.646738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.646568    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63cf5320-3ebf-4c2a-a3ce-b27e277d407c-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:38:32.799181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.799145    2581 generic.go:356] "Generic (PLEG): container finished" podID="63cf5320-3ebf-4c2a-a3ce-b27e277d407c" containerID="204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712" exitCode=0
Sep 19 09:38:32.799367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.799195    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"
Sep 19 09:38:32.799367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.799232    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg" event={"ID":"63cf5320-3ebf-4c2a-a3ce-b27e277d407c","Type":"ContainerDied","Data":"204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712"}
Sep 19 09:38:32.799367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.799280    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg" event={"ID":"63cf5320-3ebf-4c2a-a3ce-b27e277d407c","Type":"ContainerDied","Data":"048be7b3a0c34ecb0318883075a21f5c718a2b4410aa70e2226507fb79480f74"}
Sep 19 09:38:32.799367 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.799311    2581 scope.go:117] "RemoveContainer" containerID="204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712"
Sep 19 09:38:32.800712 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.800692    2581 generic.go:356] "Generic (PLEG): container finished" podID="844f2cc3-0f3a-4c0b-9015-149a1f935650" containerID="9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba" exitCode=0
Sep 19 09:38:32.800837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.800727    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"
Sep 19 09:38:32.800837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.800778    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf" event={"ID":"844f2cc3-0f3a-4c0b-9015-149a1f935650","Type":"ContainerDied","Data":"9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba"}
Sep 19 09:38:32.800837 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.800815    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf" event={"ID":"844f2cc3-0f3a-4c0b-9015-149a1f935650","Type":"ContainerDied","Data":"8335d6df9a91e7441c7dde5f648ac145c060b8d268e158c35a8f122d8795c14e"}
Sep 19 09:38:32.802273 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.802247    2581 generic.go:356] "Generic (PLEG): container finished" podID="ce478e8e-17a6-4c52-8106-283ebe4dd74d" containerID="c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5" exitCode=0
Sep 19 09:38:32.802533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.802291    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"
Sep 19 09:38:32.802533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.802338    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp" event={"ID":"ce478e8e-17a6-4c52-8106-283ebe4dd74d","Type":"ContainerDied","Data":"c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5"}
Sep 19 09:38:32.802533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.802373    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp" event={"ID":"ce478e8e-17a6-4c52-8106-283ebe4dd74d","Type":"ContainerDied","Data":"1d5d5ff76b9d3d5f8e7f8665009b6aca080343cd91aa530bb1c39f3a98ad6f94"}
Sep 19 09:38:32.804921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.804814    2581 generic.go:356] "Generic (PLEG): container finished" podID="36aa4868-42ef-4a86-ba60-bd1d18c19d06" containerID="442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a" exitCode=0
Sep 19 09:38:32.804921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.804852    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"
Sep 19 09:38:32.805087 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.804927    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm" event={"ID":"36aa4868-42ef-4a86-ba60-bd1d18c19d06","Type":"ContainerDied","Data":"442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a"}
Sep 19 09:38:32.805087 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.804962    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm" event={"ID":"36aa4868-42ef-4a86-ba60-bd1d18c19d06","Type":"ContainerDied","Data":"51d7c14bba22210e55f5398a43cff935d0b46c0589b91028224c278aa05442ea"}
Sep 19 09:38:32.806559 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.806533    2581 generic.go:356] "Generic (PLEG): container finished" podID="340bfca3-6480-43e8-9dcb-2eb7765fb1dc" containerID="49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd" exitCode=0
Sep 19 09:38:32.806678 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.806620    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67" event={"ID":"340bfca3-6480-43e8-9dcb-2eb7765fb1dc","Type":"ContainerDied","Data":"49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd"}
Sep 19 09:38:32.806678 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.806646    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67" event={"ID":"340bfca3-6480-43e8-9dcb-2eb7765fb1dc","Type":"ContainerDied","Data":"5d2db1c9072eb2cab62e028c44089d6fb412d8e13807133895c2640c036f2a17"}
Sep 19 09:38:32.806793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.806698    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"
Sep 19 09:38:32.810591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.810569    2581 scope.go:117] "RemoveContainer" containerID="204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712"
Sep 19 09:38:32.810926 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:38:32.810887    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712\": container with ID starting with 204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712 not found: ID does not exist" containerID="204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712"
Sep 19 09:38:32.811039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.810921    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712"} err="failed to get container status \"204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712\": rpc error: code = NotFound desc = could not find container \"204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712\": container with ID starting with 204cb8dc5a738bf7ba9555abc0c155a92901e2cdc13522bde72982a7c8112712 not found: ID does not exist"
Sep 19 09:38:32.811039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.810944    2581 scope.go:117] "RemoveContainer" containerID="9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba"
Sep 19 09:38:32.821500 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.821452    2581 scope.go:117] "RemoveContainer" containerID="9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba"
Sep 19 09:38:32.821794 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:38:32.821772    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba\": container with ID starting with 9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba not found: ID does not exist" containerID="9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba"
Sep 19 09:38:32.821894 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.821801    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba"} err="failed to get container status \"9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba\": rpc error: code = NotFound desc = could not find container \"9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba\": container with ID starting with 9132f5439d362ce4b2fc6a70b7b5dcf6809deb32291fdc90c92cc30551384fba not found: ID does not exist"
Sep 19 09:38:32.821894 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.821817    2581 scope.go:117] "RemoveContainer" containerID="c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5"
Sep 19 09:38:32.829976 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.829959    2581 scope.go:117] "RemoveContainer" containerID="c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5"
Sep 19 09:38:32.830281 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:38:32.830263    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5\": container with ID starting with c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5 not found: ID does not exist" containerID="c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5"
Sep 19 09:38:32.830372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.830295    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5"} err="failed to get container status \"c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5\": rpc error: code = NotFound desc = could not find container \"c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5\": container with ID starting with c0a5698cfcc363daf9b658f6cb89eb38d9d83eb6fde90a1885a8f21b31c31bb5 not found: ID does not exist"
Sep 19 09:38:32.830372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.830318    2581 scope.go:117] "RemoveContainer" containerID="442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a"
Sep 19 09:38:32.833839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.833818    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"]
Sep 19 09:38:32.837863 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.837835    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/premium-w0-14e3f6-kserve-b7c7d76ff-7ncfg"]
Sep 19 09:38:32.839277 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.839254    2581 scope.go:117] "RemoveContainer" containerID="442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a"
Sep 19 09:38:32.839609 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:38:32.839586    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a\": container with ID starting with 442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a not found: ID does not exist" containerID="442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a"
Sep 19 09:38:32.839686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.839620    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a"} err="failed to get container status \"442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a\": rpc error: code = NotFound desc = could not find container \"442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a\": container with ID starting with 442076a5df70e7e11f1635e1c512dd30cf8f473628b35aa014322fb69369447a not found: ID does not exist"
Sep 19 09:38:32.839686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.839641    2581 scope.go:117] "RemoveContainer" containerID="49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd"
Sep 19 09:38:32.847615 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.847596    2581 scope.go:117] "RemoveContainer" containerID="49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd"
Sep 19 09:38:32.847863 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:38:32.847841    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd\": container with ID starting with 49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd not found: ID does not exist" containerID="49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd"
Sep 19 09:38:32.847980 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.847864    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd"} err="failed to get container status \"49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd\": rpc error: code = NotFound desc = could not find container \"49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd\": container with ID starting with 49c1d51e1ea32183fe795aa5dcb5c966da429fc5509039c83226e4891d6b5bdd not found: ID does not exist"
Sep 19 09:38:32.853895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.853872    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"]
Sep 19 09:38:32.855793 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.855766    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-w0-14e3f6-kserve-587ff7bcc8-zkvnf"]
Sep 19 09:38:32.857667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.857634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:32.865946 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.865923    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"]
Sep 19 09:38:32.873085 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.873062    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/simulator-w0-14e3f6-kserve-57cf64cfd6-7zzhm"]
Sep 19 09:38:32.884783 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.884761    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"]
Sep 19 09:38:32.890144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.890117    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/distinct-2-w0-14e3f6-kserve-6c8d8669fc-zxg67"]
Sep 19 09:38:32.908842 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.908770    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"]
Sep 19 09:38:32.915137 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:32.915115    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-14e3f6/embedding-w0-14e3f6-kserve-564cb766c5-xf9wp"]
Sep 19 09:38:33.869597 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:33.869557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:34.343604 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:34.343569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:34.831850 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:34.831810    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="340bfca3-6480-43e8-9dcb-2eb7765fb1dc" path="/var/lib/kubelet/pods/340bfca3-6480-43e8-9dcb-2eb7765fb1dc/volumes"
Sep 19 09:38:34.832308 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:34.832292    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36aa4868-42ef-4a86-ba60-bd1d18c19d06" path="/var/lib/kubelet/pods/36aa4868-42ef-4a86-ba60-bd1d18c19d06/volumes"
Sep 19 09:38:34.832645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:34.832633    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63cf5320-3ebf-4c2a-a3ce-b27e277d407c" path="/var/lib/kubelet/pods/63cf5320-3ebf-4c2a-a3ce-b27e277d407c/volumes"
Sep 19 09:38:34.832974 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:34.832963    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="844f2cc3-0f3a-4c0b-9015-149a1f935650" path="/var/lib/kubelet/pods/844f2cc3-0f3a-4c0b-9015-149a1f935650/volumes"
Sep 19 09:38:34.833548 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:34.833532    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce478e8e-17a6-4c52-8106-283ebe4dd74d" path="/var/lib/kubelet/pods/ce478e8e-17a6-4c52-8106-283ebe4dd74d/volumes"
Sep 19 09:38:35.047778 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:35.047735    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:36.100688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:36.100647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:36.972407 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:36.972371    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:37.088233 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:37.088188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:37.333860 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:37.333764    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:37.435466 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:37.435423    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:38.565948 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:38.565889    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:39.429603 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:39.429569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:40.199189 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:40.199136    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:41.030329 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:41.030297    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:41.764958 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:41.764924    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:42.554648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:42.554613    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:43.478845 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:43.478806    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:52.369581 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:52.369542    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:38:52.698223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:52.698188    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:54.104776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:54.104740    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:54.224885 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:54.224848    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:54.944456 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:54.944413    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:55.249210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:55.249122    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:55.347690 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:55.347655    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:55.885712 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:55.885678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:56.062214 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:56.062168    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:56.550701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:56.550664    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:57.245357 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:57.245302    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:59.434622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:59.434585    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:59.536654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:59.536623    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:38:59.675908 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:38:59.675877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:00.751602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:00.751568    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:01.489422 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:01.489388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:02.171727 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:02.171688    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:03.004584 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:03.004541    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:03.679698 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:03.679662    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:04.608144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:04.608084    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:09.481526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:09.481489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:39:11.150435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:11.150398    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:15.500356 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.500321    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"]
Sep 19 09:39:15.500948 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.500526    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8" podUID="bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" containerName="maas-api" containerID="cri-o://a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7" gracePeriod=30
Sep 19 09:39:15.532471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.532419    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:15.562623 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.562586    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"]
Sep 19 09:39:15.562879 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.562840    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs" podUID="03df881e-d96f-42f1-ad66-0cb1f5add5a7" containerName="payload-processing" containerID="cri-o://fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c" gracePeriod=30
Sep 19 09:39:15.633712 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.633676    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:15.733838 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.733798    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:15.809289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.809253    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:15.811129 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.811112    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:39:15.836804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.836780    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-maas-api-tls\") pod \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") "
Sep 19 09:39:15.836939 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.836851    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdmvg\" (UniqueName: \"kubernetes.io/projected/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-kube-api-access-sdmvg\") pod \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") "
Sep 19 09:39:15.837450 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.837427    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-openshift-service-ca\") pod \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") "
Sep 19 09:39:15.837652 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.837468    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-metrics-certs\") pod \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\" (UID: \"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef\") "
Sep 19 09:39:15.837845 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.837812    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" (UID: "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:39:15.839611 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.839576    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-kube-api-access-sdmvg" (OuterVolumeSpecName: "kube-api-access-sdmvg") pod "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" (UID: "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef"). InnerVolumeSpecName "kube-api-access-sdmvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:39:15.839702 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.839615    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" (UID: "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:39:15.840040 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.840017    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" (UID: "bff5ec2c-0445-4e74-ac9a-68ac3f0967ef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:39:15.843792 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.843774    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:39:15.938242 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.938209    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03df881e-d96f-42f1-ad66-0cb1f5add5a7-plugins-config-volume\") pod \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") "
Sep 19 09:39:15.938420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.938305    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcrc6\" (UniqueName: \"kubernetes.io/projected/03df881e-d96f-42f1-ad66-0cb1f5add5a7-kube-api-access-tcrc6\") pod \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\" (UID: \"03df881e-d96f-42f1-ad66-0cb1f5add5a7\") "
Sep 19 09:39:15.938485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.938464    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:15.938485 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.938478    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdmvg\" (UniqueName: \"kubernetes.io/projected/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-kube-api-access-sdmvg\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:15.938555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.938487    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:15.938555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.938495    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:15.938636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.938565    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03df881e-d96f-42f1-ad66-0cb1f5add5a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "03df881e-d96f-42f1-ad66-0cb1f5add5a7" (UID: "03df881e-d96f-42f1-ad66-0cb1f5add5a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:39:15.940514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:15.940492    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03df881e-d96f-42f1-ad66-0cb1f5add5a7-kube-api-access-tcrc6" (OuterVolumeSpecName: "kube-api-access-tcrc6") pod "03df881e-d96f-42f1-ad66-0cb1f5add5a7" (UID: "03df881e-d96f-42f1-ad66-0cb1f5add5a7"). InnerVolumeSpecName "kube-api-access-tcrc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:39:16.032902 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.032816    2581 generic.go:356] "Generic (PLEG): container finished" podID="bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" containerID="a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7" exitCode=0
Sep 19 09:39:16.032902 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.032863    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"
Sep 19 09:39:16.033134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.032894    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8" event={"ID":"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef","Type":"ContainerDied","Data":"a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7"}
Sep 19 09:39:16.033134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.032953    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8" event={"ID":"bff5ec2c-0445-4e74-ac9a-68ac3f0967ef","Type":"ContainerDied","Data":"aa67d64cd183efafe4ab37dddb89d30e8db8fc4228927118aa74885a316eb4dc"}
Sep 19 09:39:16.033134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.032981    2581 scope.go:117] "RemoveContainer" containerID="a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7"
Sep 19 09:39:16.034389 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.034364    2581 generic.go:356] "Generic (PLEG): container finished" podID="03df881e-d96f-42f1-ad66-0cb1f5add5a7" containerID="fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c" exitCode=0
Sep 19 09:39:16.034487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.034423    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"
Sep 19 09:39:16.034487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.034413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs" event={"ID":"03df881e-d96f-42f1-ad66-0cb1f5add5a7","Type":"ContainerDied","Data":"fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c"}
Sep 19 09:39:16.034487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.034466    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs" event={"ID":"03df881e-d96f-42f1-ad66-0cb1f5add5a7","Type":"ContainerDied","Data":"ac084e4fb83156c680ba11408db93c0538d603accaa382f3cf52100f7daa711f"}
Sep 19 09:39:16.038922 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.038898    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcrc6\" (UniqueName: \"kubernetes.io/projected/03df881e-d96f-42f1-ad66-0cb1f5add5a7-kube-api-access-tcrc6\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:16.038922 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.038923    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/03df881e-d96f-42f1-ad66-0cb1f5add5a7-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:16.045948 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.045923    2581 scope.go:117] "RemoveContainer" containerID="a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7"
Sep 19 09:39:16.046313 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:39:16.046296    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7\": container with ID starting with a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7 not found: ID does not exist" containerID="a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7"
Sep 19 09:39:16.046382 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.046323    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7"} err="failed to get container status \"a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7\": rpc error: code = NotFound desc = could not find container \"a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7\": container with ID starting with a6428518fd572461efcd5395e5184a22dba9e73206bbe07d0e1257f4cfff6eb7 not found: ID does not exist"
Sep 19 09:39:16.046382 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.046341    2581 scope.go:117] "RemoveContainer" containerID="fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c"
Sep 19 09:39:16.054750 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.054734    2581 scope.go:117] "RemoveContainer" containerID="fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c"
Sep 19 09:39:16.055010 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:39:16.054992    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c\": container with ID starting with fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c not found: ID does not exist" containerID="fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c"
Sep 19 09:39:16.055060 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.055015    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c"} err="failed to get container status \"fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c\": rpc error: code = NotFound desc = could not find container \"fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c\": container with ID starting with fb30144d5c022c003bd8a519af3d708212b2f7a494335b024eb9bf2337073c9c not found: ID does not exist"
Sep 19 09:39:16.060258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.060235    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"]
Sep 19 09:39:16.063418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.063398    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0304ec-5f5c685654-kjxj8"]
Sep 19 09:39:16.076186 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.076164    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"]
Sep 19 09:39:16.085683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.085659    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0304ec-56ffcf446c-gfpjs"]
Sep 19 09:39:16.109723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.109693    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:16.180933 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.180898    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:16.830547 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.830470    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03df881e-d96f-42f1-ad66-0cb1f5add5a7" path="/var/lib/kubelet/pods/03df881e-d96f-42f1-ad66-0cb1f5add5a7/volumes"
Sep 19 09:39:16.830900 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:16.830840    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" path="/var/lib/kubelet/pods/bff5ec2c-0445-4e74-ac9a-68ac3f0967ef/volumes"
Sep 19 09:39:17.009534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:17.009500    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:17.794738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:17.794646    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:18.619822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:18.619787    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:19.581580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:19.581550    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:20.314491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:20.314459    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:22.075259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:22.075222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:22.221890 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:22.221860    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:22.400315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:22.400281    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:23.111766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:23.111729    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:24.007292 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:24.007252    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:25.216677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:25.216643    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:25.932549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:25.932517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:26.672351 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:26.672312    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:39:26.805940 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:26.805902    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:27.121460 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:27.121373    2581 scope.go:117] "RemoveContainer" containerID="37f93a2f0655fa9324a57576b58146f1fbc6fcbeb9f1032b37e12d45adb8e5a0"
Sep 19 09:39:27.508372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:27.508333    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:33.421037 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.421004    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"]
Sep 19 09:39:33.421500 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.421211    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh" podUID="e127e811-65b3-4e6b-beae-51c11489220a" containerName="maas-api" containerID="cri-o://2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4" gracePeriod=30
Sep 19 09:39:33.496477 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.496441    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:33.650170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.650135    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:33.682245 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.682176    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:39:33.771554 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.771519    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:33.803305 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.803253    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-maas-api-tls\") pod \"e127e811-65b3-4e6b-beae-51c11489220a\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") "
Sep 19 09:39:33.803495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.803368    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-metrics-certs\") pod \"e127e811-65b3-4e6b-beae-51c11489220a\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") "
Sep 19 09:39:33.803573 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.803551    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpq5c\" (UniqueName: \"kubernetes.io/projected/e127e811-65b3-4e6b-beae-51c11489220a-kube-api-access-qpq5c\") pod \"e127e811-65b3-4e6b-beae-51c11489220a\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") "
Sep 19 09:39:33.803656 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.803644    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e127e811-65b3-4e6b-beae-51c11489220a-openshift-service-ca\") pod \"e127e811-65b3-4e6b-beae-51c11489220a\" (UID: \"e127e811-65b3-4e6b-beae-51c11489220a\") "
Sep 19 09:39:33.804147 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.804086    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e127e811-65b3-4e6b-beae-51c11489220a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e127e811-65b3-4e6b-beae-51c11489220a" (UID: "e127e811-65b3-4e6b-beae-51c11489220a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:39:33.805890 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.805865    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e127e811-65b3-4e6b-beae-51c11489220a" (UID: "e127e811-65b3-4e6b-beae-51c11489220a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:39:33.805998 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.805913    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e127e811-65b3-4e6b-beae-51c11489220a" (UID: "e127e811-65b3-4e6b-beae-51c11489220a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:39:33.805998 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.805961    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e127e811-65b3-4e6b-beae-51c11489220a-kube-api-access-qpq5c" (OuterVolumeSpecName: "kube-api-access-qpq5c") pod "e127e811-65b3-4e6b-beae-51c11489220a" (UID: "e127e811-65b3-4e6b-beae-51c11489220a"). InnerVolumeSpecName "kube-api-access-qpq5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:39:33.889960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.889923    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:33.904680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.904647    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpq5c\" (UniqueName: \"kubernetes.io/projected/e127e811-65b3-4e6b-beae-51c11489220a-kube-api-access-qpq5c\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:33.904680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.904681    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e127e811-65b3-4e6b-beae-51c11489220a-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:33.904898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.904695    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:33.904898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.904706    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e127e811-65b3-4e6b-beae-51c11489220a-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:33.988522 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:33.988425    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:34.127883 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.127847    2581 generic.go:356] "Generic (PLEG): container finished" podID="e127e811-65b3-4e6b-beae-51c11489220a" containerID="2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4" exitCode=0
Sep 19 09:39:34.128083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.127911    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"
Sep 19 09:39:34.128083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.127928    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh" event={"ID":"e127e811-65b3-4e6b-beae-51c11489220a","Type":"ContainerDied","Data":"2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4"}
Sep 19 09:39:34.128083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.127981    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh" event={"ID":"e127e811-65b3-4e6b-beae-51c11489220a","Type":"ContainerDied","Data":"41d12f51cfc02901d0058dbca39063ba139074ffd9142aa320b22e6f2b925ba9"}
Sep 19 09:39:34.128083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.128007    2581 scope.go:117] "RemoveContainer" containerID="2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4"
Sep 19 09:39:34.137223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.137197    2581 scope.go:117] "RemoveContainer" containerID="2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4"
Sep 19 09:39:34.137501 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:39:34.137482    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4\": container with ID starting with 2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4 not found: ID does not exist" containerID="2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4"
Sep 19 09:39:34.137568 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.137515    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4"} err="failed to get container status \"2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4\": rpc error: code = NotFound desc = could not find container \"2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4\": container with ID starting with 2eda8d6a5c2f4d18f096e16c81a27bcd70f8490cf725c41a068702dd8ee855a4 not found: ID does not exist"
Sep 19 09:39:34.150372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.150346    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"]
Sep 19 09:39:34.157355 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.157332    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-261ce4-77f999d594-prqfh"]
Sep 19 09:39:34.285317 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.285234    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:34.391871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.391829    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:34.831765 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:34.831730    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e127e811-65b3-4e6b-beae-51c11489220a" path="/var/lib/kubelet/pods/e127e811-65b3-4e6b-beae-51c11489220a/volumes"
Sep 19 09:39:35.639399 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:35.639368    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:36.522785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:36.522744    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:37.225949 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:37.225921    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:40.366127 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:40.366072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:40.471137 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:40.471073    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:40.665326 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:40.665296    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:41.288614 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:41.288579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:42.220355 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:42.220313    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:42.924705 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:42.924663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:43.805674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:43.805644    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:44.616624 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:44.616585    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:45.274203 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:45.274164    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:39:46.698746 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.698712    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7848-e76345"]
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699055    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36aa4868-42ef-4a86-ba60-bd1d18c19d06" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699065    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="36aa4868-42ef-4a86-ba60-bd1d18c19d06" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699075    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="844f2cc3-0f3a-4c0b-9015-149a1f935650" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699081    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="844f2cc3-0f3a-4c0b-9015-149a1f935650" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699090    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="340bfca3-6480-43e8-9dcb-2eb7765fb1dc" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699110    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="340bfca3-6480-43e8-9dcb-2eb7765fb1dc" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699117    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e127e811-65b3-4e6b-beae-51c11489220a" containerName="maas-api"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699123    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="e127e811-65b3-4e6b-beae-51c11489220a" containerName="maas-api"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699136    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cf5320-3ebf-4c2a-a3ce-b27e277d407c" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699141    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cf5320-3ebf-4c2a-a3ce-b27e277d407c" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699150    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" containerName="maas-api"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699155    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" containerName="maas-api"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699161    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03df881e-d96f-42f1-ad66-0cb1f5add5a7" containerName="payload-processing"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699167    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="03df881e-d96f-42f1-ad66-0cb1f5add5a7" containerName="payload-processing"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699179    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce478e8e-17a6-4c52-8106-283ebe4dd74d" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699184    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce478e8e-17a6-4c52-8106-283ebe4dd74d" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699238    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="03df881e-d96f-42f1-ad66-0cb1f5add5a7" containerName="payload-processing"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699245    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce478e8e-17a6-4c52-8106-283ebe4dd74d" containerName="main"
Sep 19 09:39:46.699243 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699252    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="340bfca3-6480-43e8-9dcb-2eb7765fb1dc" containerName="main"
Sep 19 09:39:46.699823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699258    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="844f2cc3-0f3a-4c0b-9015-149a1f935650" containerName="main"
Sep 19 09:39:46.699823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699264    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="36aa4868-42ef-4a86-ba60-bd1d18c19d06" containerName="main"
Sep 19 09:39:46.699823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699269    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bff5ec2c-0445-4e74-ac9a-68ac3f0967ef" containerName="maas-api"
Sep 19 09:39:46.699823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699275    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="e127e811-65b3-4e6b-beae-51c11489220a" containerName="maas-api"
Sep 19 09:39:46.699823 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.699282    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="63cf5320-3ebf-4c2a-a3ce-b27e277d407c" containerName="main"
Sep 19 09:39:46.703656 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.703638    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:46.710919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.710889    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q5mhq\""
Sep 19 09:39:46.712580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.712556    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7848-e76345"]
Sep 19 09:39:46.814007 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.813961    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fz9t\" (UniqueName: \"kubernetes.io/projected/1e038db0-9d31-4208-a97d-30ee919adb1a-kube-api-access-7fz9t\") pod \"test-curl-7848-e76345\" (UID: \"1e038db0-9d31-4208-a97d-30ee919adb1a\") " pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:46.915545 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.915507    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fz9t\" (UniqueName: \"kubernetes.io/projected/1e038db0-9d31-4208-a97d-30ee919adb1a-kube-api-access-7fz9t\") pod \"test-curl-7848-e76345\" (UID: \"1e038db0-9d31-4208-a97d-30ee919adb1a\") " pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:46.923842 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:46.923807    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fz9t\" (UniqueName: \"kubernetes.io/projected/1e038db0-9d31-4208-a97d-30ee919adb1a-kube-api-access-7fz9t\") pod \"test-curl-7848-e76345\" (UID: \"1e038db0-9d31-4208-a97d-30ee919adb1a\") " pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:47.013494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:47.013392    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:47.286806 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:47.286776    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7848-e76345"]
Sep 19 09:39:47.289595 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:39:47.289566    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e038db0_9d31_4208_a97d_30ee919adb1a.slice/crio-83751f85ef448158b0042e03c7b1b95c316d55ba1526af021f41209b56b3205d WatchSource:0}: Error finding container 83751f85ef448158b0042e03c7b1b95c316d55ba1526af021f41209b56b3205d: Status 404 returned error can't find the container with id 83751f85ef448158b0042e03c7b1b95c316d55ba1526af021f41209b56b3205d
Sep 19 09:39:48.197451 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:48.197411    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7848-e76345" event={"ID":"1e038db0-9d31-4208-a97d-30ee919adb1a","Type":"ContainerStarted","Data":"91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0"}
Sep 19 09:39:48.197451 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:48.197463    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7848-e76345" event={"ID":"1e038db0-9d31-4208-a97d-30ee919adb1a","Type":"ContainerStarted","Data":"83751f85ef448158b0042e03c7b1b95c316d55ba1526af021f41209b56b3205d"}
Sep 19 09:39:48.217871 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:48.217820    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7848-e76345" podStartSLOduration=2.217804046 podStartE2EDuration="2.217804046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:39:48.216368911 +0000 UTC m=+2001.903174209" watchObservedRunningTime="2026-09-19 09:39:48.217804046 +0000 UTC m=+2001.904609343"
Sep 19 09:39:48.870365 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:48.870331    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7848-e76345"]
Sep 19 09:39:48.874511 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:48.874487    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7848-e76345"]
Sep 19 09:39:49.202678 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:39:49.202649    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7848-e76345" container="test-curl-7848-e76345"
Sep 19 09:39:50.197813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:50.197778    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"]
Sep 19 09:39:50.208178 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:50.208125    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7848-e76345" podUID="1e038db0-9d31-4208-a97d-30ee919adb1a" containerName="test-curl-7848-e76345" containerID="cri-o://91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0" gracePeriod=2
Sep 19 09:39:50.218708 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:39:50.218674    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7848-e76345\" is forbidden: User \"system:node:ip-10-0-131-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-218.ec2.internal' and this object" podUID="1e038db0-9d31-4208-a97d-30ee919adb1a" pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:50.230423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:50.230393    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:51.278437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.278406    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"]
Sep 19 09:39:51.292494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.292465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:51.505719 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.505687    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"]
Sep 19 09:39:51.505916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.505884    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:51.505916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.505902    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8" podUID="d3c9e0b3-544d-4631-aaa2-77e88bc45d05" containerName="maas-api" containerID="cri-o://04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e" gracePeriod=30
Sep 19 09:39:51.602409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.602327    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:51.683407 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.683323    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:51.748517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.748494    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:39:51.759972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.759943    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-openshift-service-ca\") pod \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") "
Sep 19 09:39:51.760144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.759982    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5qcl\" (UniqueName: \"kubernetes.io/projected/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-kube-api-access-z5qcl\") pod \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") "
Sep 19 09:39:51.760144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.760023    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls\") pod \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") "
Sep 19 09:39:51.760144 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.760068    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-metrics-certs\") pod \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\" (UID: \"d3c9e0b3-544d-4631-aaa2-77e88bc45d05\") "
Sep 19 09:39:51.760405 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.760377    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d3c9e0b3-544d-4631-aaa2-77e88bc45d05" (UID: "d3c9e0b3-544d-4631-aaa2-77e88bc45d05"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:39:51.762452 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.762422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d3c9e0b3-544d-4631-aaa2-77e88bc45d05" (UID: "d3c9e0b3-544d-4631-aaa2-77e88bc45d05"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:39:51.762589 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.762564    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d3c9e0b3-544d-4631-aaa2-77e88bc45d05" (UID: "d3c9e0b3-544d-4631-aaa2-77e88bc45d05"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:39:51.762632 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.762574    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-kube-api-access-z5qcl" (OuterVolumeSpecName: "kube-api-access-z5qcl") pod "d3c9e0b3-544d-4631-aaa2-77e88bc45d05" (UID: "d3c9e0b3-544d-4631-aaa2-77e88bc45d05"). InnerVolumeSpecName "kube-api-access-z5qcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:39:51.777023 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.776965    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:51.861012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.860910    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:51.861012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.860957    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5qcl\" (UniqueName: \"kubernetes.io/projected/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-kube-api-access-z5qcl\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:51.861012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.860968    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:51.861012 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.860977    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c9e0b3-544d-4631-aaa2-77e88bc45d05-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:51.869452 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:51.869411    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:52.111963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.111862    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:52.214954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.214784    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:52.219748 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.219722    2581 generic.go:356] "Generic (PLEG): container finished" podID="d3c9e0b3-544d-4631-aaa2-77e88bc45d05" containerID="04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e" exitCode=0
Sep 19 09:39:52.219850 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.219769    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"
Sep 19 09:39:52.219850 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.219788    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8" event={"ID":"d3c9e0b3-544d-4631-aaa2-77e88bc45d05","Type":"ContainerDied","Data":"04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e"}
Sep 19 09:39:52.219850 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.219841    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8" event={"ID":"d3c9e0b3-544d-4631-aaa2-77e88bc45d05","Type":"ContainerDied","Data":"9ddd7751b2dcf91c6dc537c927d2f42375fb17113e1a2bcd3b17fe8073351ba3"}
Sep 19 09:39:52.219957 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.219871    2581 scope.go:117] "RemoveContainer" containerID="04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e"
Sep 19 09:39:52.276023 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.275999    2581 scope.go:117] "RemoveContainer" containerID="04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e"
Sep 19 09:39:52.276468 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:39:52.276442    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e\": container with ID starting with 04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e not found: ID does not exist" containerID="04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e"
Sep 19 09:39:52.276532 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.276481    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e"} err="failed to get container status \"04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e\": rpc error: code = NotFound desc = could not find container \"04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e\": container with ID starting with 04d4db5c2c51a1fe5b253dfd09d4975312341ec2a229c59720cbd39cc650fb8e not found: ID does not exist"
Sep 19 09:39:52.296808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.296778    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"]
Sep 19 09:39:52.304715 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.304691    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-66c0fb-7fcff44f47-rgvm8"]
Sep 19 09:39:52.372632 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.372606    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:52.464181 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.464130    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fz9t\" (UniqueName: \"kubernetes.io/projected/1e038db0-9d31-4208-a97d-30ee919adb1a-kube-api-access-7fz9t\") pod \"1e038db0-9d31-4208-a97d-30ee919adb1a\" (UID: \"1e038db0-9d31-4208-a97d-30ee919adb1a\") "
Sep 19 09:39:52.466499 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.466469    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e038db0-9d31-4208-a97d-30ee919adb1a-kube-api-access-7fz9t" (OuterVolumeSpecName: "kube-api-access-7fz9t") pod "1e038db0-9d31-4208-a97d-30ee919adb1a" (UID: "1e038db0-9d31-4208-a97d-30ee919adb1a"). InnerVolumeSpecName "kube-api-access-7fz9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:39:52.565001 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.564953    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fz9t\" (UniqueName: \"kubernetes.io/projected/1e038db0-9d31-4208-a97d-30ee919adb1a-kube-api-access-7fz9t\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:39:52.830533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.830499    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e038db0-9d31-4208-a97d-30ee919adb1a" path="/var/lib/kubelet/pods/1e038db0-9d31-4208-a97d-30ee919adb1a/volumes"
Sep 19 09:39:52.830898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:52.830885    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3c9e0b3-544d-4631-aaa2-77e88bc45d05" path="/var/lib/kubelet/pods/d3c9e0b3-544d-4631-aaa2-77e88bc45d05/volumes"
Sep 19 09:39:53.120112 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:53.120010    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:53.226768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:53.226728    2581 generic.go:356] "Generic (PLEG): container finished" podID="1e038db0-9d31-4208-a97d-30ee919adb1a" containerID="91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0" exitCode=137
Sep 19 09:39:53.226768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:53.226771    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-e76345"
Sep 19 09:39:53.226990 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:53.226789    2581 scope.go:117] "RemoveContainer" containerID="91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0"
Sep 19 09:39:53.235260 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:53.235237    2581 scope.go:117] "RemoveContainer" containerID="91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0"
Sep 19 09:39:53.235556 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:39:53.235536    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0\": container with ID starting with 91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0 not found: ID does not exist" containerID="91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0"
Sep 19 09:39:53.235602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:53.235566    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0"} err="failed to get container status \"91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0\": rpc error: code = NotFound desc = could not find container \"91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0\": container with ID starting with 91336c78dab52d3ef4d6345431aa3752073b132cc7ec7811b27d854893b65db0 not found: ID does not exist"
Sep 19 09:39:53.822155 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:53.822118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:55.546335 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:55.546301    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:39:58.096149 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:58.096088    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:58.175919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:58.175877    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:58.357840 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:58.357739    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:59.888771 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.888735    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"]
Sep 19 09:39:59.889177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.889051    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c9e0b3-544d-4631-aaa2-77e88bc45d05" containerName="maas-api"
Sep 19 09:39:59.889177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.889063    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c9e0b3-544d-4631-aaa2-77e88bc45d05" containerName="maas-api"
Sep 19 09:39:59.889177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.889078    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e038db0-9d31-4208-a97d-30ee919adb1a" containerName="test-curl-7848-e76345"
Sep 19 09:39:59.889177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.889083    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e038db0-9d31-4208-a97d-30ee919adb1a" containerName="test-curl-7848-e76345"
Sep 19 09:39:59.889177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.889152    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c9e0b3-544d-4631-aaa2-77e88bc45d05" containerName="maas-api"
Sep 19 09:39:59.889177 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.889163    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e038db0-9d31-4208-a97d-30ee919adb1a" containerName="test-curl-7848-e76345"
Sep 19 09:39:59.891988 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.891971    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:39:59.894754 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.894728    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3240ec\"/\"sub-test-model-3240ec-kserve-self-signed-certs\""
Sep 19 09:39:59.908333 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.908307    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"]
Sep 19 09:39:59.926409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.926379    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:39:59.930397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.930373    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-home\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:39:59.930524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.930404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-dshm\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:39:59.930524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.930436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tmp-dir\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:39:59.930524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.930499    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztnhm\" (UniqueName: \"kubernetes.io/projected/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-kube-api-access-ztnhm\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:39:59.930651 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.930558    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-model-cache\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:39:59.930651 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:39:59.930587    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tls-certs\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031338 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031291    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztnhm\" (UniqueName: \"kubernetes.io/projected/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-kube-api-access-ztnhm\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-model-cache\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031403    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tls-certs\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031473    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-home\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031493    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-dshm\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031765 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031539    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tmp-dir\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031843 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031817    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-model-cache\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031843 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031835    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-home\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.031951 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.031900    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tmp-dir\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.034056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.034035    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-dshm\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.034319 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.034300    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tls-certs\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.040975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.040946    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztnhm\" (UniqueName: \"kubernetes.io/projected/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-kube-api-access-ztnhm\") pod \"sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.201501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.201393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:00.478946 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:00.478914    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"]
Sep 19 09:40:00.481549 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:40:00.481522    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c84f7d3_6f41_4de1_9e4c_6f28f7878848.slice/crio-afc310769d9c89df0c6d40b23062c51ba8d4ef0f01dc2a42c457db7216606769 WatchSource:0}: Error finding container afc310769d9c89df0c6d40b23062c51ba8d4ef0f01dc2a42c457db7216606769: Status 404 returned error can't find the container with id afc310769d9c89df0c6d40b23062c51ba8d4ef0f01dc2a42c457db7216606769
Sep 19 09:40:01.269622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:01.269587    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4" event={"ID":"7c84f7d3-6f41-4de1-9e4c-6f28f7878848","Type":"ContainerStarted","Data":"7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986"}
Sep 19 09:40:01.269622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:01.269632    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4" event={"ID":"7c84f7d3-6f41-4de1-9e4c-6f28f7878848","Type":"ContainerStarted","Data":"afc310769d9c89df0c6d40b23062c51ba8d4ef0f01dc2a42c457db7216606769"}
Sep 19 09:40:01.290337 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:01.290286    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4" podStartSLOduration=2.290271297 podStartE2EDuration="2.290271297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:40:01.288917879 +0000 UTC m=+2014.975723177" watchObservedRunningTime="2026-09-19 09:40:01.290271297 +0000 UTC m=+2014.977076628"
Sep 19 09:40:05.218134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.218070    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw" podUID="95df44b9-143b-4f44-8510-85739e72fb74" containerName="main" containerID="cri-o://0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006" gracePeriod=45
Sep 19 09:40:05.467213 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.467187    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:40:05.580182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580063    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-home\") pod \"95df44b9-143b-4f44-8510-85739e72fb74\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") "
Sep 19 09:40:05.580182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580136    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h695x\" (UniqueName: \"kubernetes.io/projected/95df44b9-143b-4f44-8510-85739e72fb74-kube-api-access-h695x\") pod \"95df44b9-143b-4f44-8510-85739e72fb74\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") "
Sep 19 09:40:05.580412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580234    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-model-cache\") pod \"95df44b9-143b-4f44-8510-85739e72fb74\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") "
Sep 19 09:40:05.580412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580275    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-tmp-dir\") pod \"95df44b9-143b-4f44-8510-85739e72fb74\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") "
Sep 19 09:40:05.580412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580303    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-dshm\") pod \"95df44b9-143b-4f44-8510-85739e72fb74\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") "
Sep 19 09:40:05.580412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580327    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95df44b9-143b-4f44-8510-85739e72fb74-tls-certs\") pod \"95df44b9-143b-4f44-8510-85739e72fb74\" (UID: \"95df44b9-143b-4f44-8510-85739e72fb74\") "
Sep 19 09:40:05.580412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580372    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-home" (OuterVolumeSpecName: "home") pod "95df44b9-143b-4f44-8510-85739e72fb74" (UID: "95df44b9-143b-4f44-8510-85739e72fb74"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:05.580648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-model-cache" (OuterVolumeSpecName: "model-cache") pod "95df44b9-143b-4f44-8510-85739e72fb74" (UID: "95df44b9-143b-4f44-8510-85739e72fb74"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:05.580648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580545    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "95df44b9-143b-4f44-8510-85739e72fb74" (UID: "95df44b9-143b-4f44-8510-85739e72fb74"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:05.580648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580635    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:05.580768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580655    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:05.580768 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.580672    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:05.582644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.582621    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-dshm" (OuterVolumeSpecName: "dshm") pod "95df44b9-143b-4f44-8510-85739e72fb74" (UID: "95df44b9-143b-4f44-8510-85739e72fb74"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:05.582742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.582668    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95df44b9-143b-4f44-8510-85739e72fb74-kube-api-access-h695x" (OuterVolumeSpecName: "kube-api-access-h695x") pod "95df44b9-143b-4f44-8510-85739e72fb74" (UID: "95df44b9-143b-4f44-8510-85739e72fb74"). InnerVolumeSpecName "kube-api-access-h695x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:40:05.582742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.582725    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95df44b9-143b-4f44-8510-85739e72fb74-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "95df44b9-143b-4f44-8510-85739e72fb74" (UID: "95df44b9-143b-4f44-8510-85739e72fb74"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:40:05.681916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.681871    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h695x\" (UniqueName: \"kubernetes.io/projected/95df44b9-143b-4f44-8510-85739e72fb74-kube-api-access-h695x\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:05.681916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.681907    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95df44b9-143b-4f44-8510-85739e72fb74-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:05.681916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:05.681917    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95df44b9-143b-4f44-8510-85739e72fb74-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:06.295244 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.295205    2581 generic.go:356] "Generic (PLEG): container finished" podID="95df44b9-143b-4f44-8510-85739e72fb74" containerID="0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006" exitCode=0
Sep 19 09:40:06.295708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.295257    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"
Sep 19 09:40:06.295708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.295291    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw" event={"ID":"95df44b9-143b-4f44-8510-85739e72fb74","Type":"ContainerDied","Data":"0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006"}
Sep 19 09:40:06.295708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.295335    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw" event={"ID":"95df44b9-143b-4f44-8510-85739e72fb74","Type":"ContainerDied","Data":"ed2c90ae4cdb165483962a7a5a162e65b99898ec12fbe8e8ee59664e56ead405"}
Sep 19 09:40:06.295708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.295352    2581 scope.go:117] "RemoveContainer" containerID="0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006"
Sep 19 09:40:06.297884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.297838    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5" podUID="16369e5b-22ee-4368-b364-37820c889fcc" containerName="main" containerID="cri-o://0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834" gracePeriod=45
Sep 19 09:40:06.304963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.304945    2581 scope.go:117] "RemoveContainer" containerID="0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006"
Sep 19 09:40:06.305363 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:40:06.305337    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006\": container with ID starting with 0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006 not found: ID does not exist" containerID="0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006"
Sep 19 09:40:06.305465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.305371    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006"} err="failed to get container status \"0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006\": rpc error: code = NotFound desc = could not find container \"0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006\": container with ID starting with 0bfcd7bc8c5b3243c770dc632fbcceb1eef3b7cae76009a1c0d0cfcc728c0006 not found: ID does not exist"
Sep 19 09:40:06.317756 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.317730    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"]
Sep 19 09:40:06.321035 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.321009    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/auth-test-model-3240ec-kserve-589f79c8c-rjkqw"]
Sep 19 09:40:06.551039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.550977    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:40:06.587655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.587619    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-home\") pod \"16369e5b-22ee-4368-b364-37820c889fcc\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") "
Sep 19 09:40:06.587857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.587681    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-dshm\") pod \"16369e5b-22ee-4368-b364-37820c889fcc\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") "
Sep 19 09:40:06.587857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.587729    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-tmp-dir\") pod \"16369e5b-22ee-4368-b364-37820c889fcc\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") "
Sep 19 09:40:06.587857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.587753    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7vjb\" (UniqueName: \"kubernetes.io/projected/16369e5b-22ee-4368-b364-37820c889fcc-kube-api-access-p7vjb\") pod \"16369e5b-22ee-4368-b364-37820c889fcc\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") "
Sep 19 09:40:06.587857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.587774    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16369e5b-22ee-4368-b364-37820c889fcc-tls-certs\") pod \"16369e5b-22ee-4368-b364-37820c889fcc\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") "
Sep 19 09:40:06.587857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.587805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-model-cache\") pod \"16369e5b-22ee-4368-b364-37820c889fcc\" (UID: \"16369e5b-22ee-4368-b364-37820c889fcc\") "
Sep 19 09:40:06.588170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.587959    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-home" (OuterVolumeSpecName: "home") pod "16369e5b-22ee-4368-b364-37820c889fcc" (UID: "16369e5b-22ee-4368-b364-37820c889fcc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:06.588170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.588034    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "16369e5b-22ee-4368-b364-37820c889fcc" (UID: "16369e5b-22ee-4368-b364-37820c889fcc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:06.588283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.588212    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-model-cache" (OuterVolumeSpecName: "model-cache") pod "16369e5b-22ee-4368-b364-37820c889fcc" (UID: "16369e5b-22ee-4368-b364-37820c889fcc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:06.590323 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.590292    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16369e5b-22ee-4368-b364-37820c889fcc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "16369e5b-22ee-4368-b364-37820c889fcc" (UID: "16369e5b-22ee-4368-b364-37820c889fcc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:40:06.590443 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.590402    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16369e5b-22ee-4368-b364-37820c889fcc-kube-api-access-p7vjb" (OuterVolumeSpecName: "kube-api-access-p7vjb") pod "16369e5b-22ee-4368-b364-37820c889fcc" (UID: "16369e5b-22ee-4368-b364-37820c889fcc"). InnerVolumeSpecName "kube-api-access-p7vjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:40:06.590443 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.590434    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-dshm" (OuterVolumeSpecName: "dshm") pod "16369e5b-22ee-4368-b364-37820c889fcc" (UID: "16369e5b-22ee-4368-b364-37820c889fcc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:40:06.688989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.688946    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:06.688989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.688978    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7vjb\" (UniqueName: \"kubernetes.io/projected/16369e5b-22ee-4368-b364-37820c889fcc-kube-api-access-p7vjb\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:06.688989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.688988    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16369e5b-22ee-4368-b364-37820c889fcc-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:06.688989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.688997    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:06.688989 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.689005    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:06.689331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.689012    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16369e5b-22ee-4368-b364-37820c889fcc-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:40:06.831707 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:06.831615    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95df44b9-143b-4f44-8510-85739e72fb74" path="/var/lib/kubelet/pods/95df44b9-143b-4f44-8510-85739e72fb74/volumes"
Sep 19 09:40:07.301124 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.301069    2581 generic.go:356] "Generic (PLEG): container finished" podID="16369e5b-22ee-4368-b364-37820c889fcc" containerID="0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834" exitCode=0
Sep 19 09:40:07.301543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.301137    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"
Sep 19 09:40:07.301543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.301142    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5" event={"ID":"16369e5b-22ee-4368-b364-37820c889fcc","Type":"ContainerDied","Data":"0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834"}
Sep 19 09:40:07.301543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.301190    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5" event={"ID":"16369e5b-22ee-4368-b364-37820c889fcc","Type":"ContainerDied","Data":"f7c8c4fde43ce723075a7f203f81618e51a07008271cc761f56a22cc21e91e2e"}
Sep 19 09:40:07.301543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.301211    2581 scope.go:117] "RemoveContainer" containerID="0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834"
Sep 19 09:40:07.310042 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.310023    2581 scope.go:117] "RemoveContainer" containerID="0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834"
Sep 19 09:40:07.310384 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:40:07.310363    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834\": container with ID starting with 0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834 not found: ID does not exist" containerID="0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834"
Sep 19 09:40:07.310442 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.310393    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834"} err="failed to get container status \"0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834\": rpc error: code = NotFound desc = could not find container \"0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834\": container with ID starting with 0b67c8c55d62e19890a520f7b443e639ecff14fff314b1b035e6fe12f25e0834 not found: ID does not exist"
Sep 19 09:40:07.321893 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.321862    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"]
Sep 19 09:40:07.326504 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:07.326482    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/auth-test-model-1afa13-kserve-7948c89dd7-66hh5"]
Sep 19 09:40:08.831070 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:08.831036    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16369e5b-22ee-4368-b364-37820c889fcc" path="/var/lib/kubelet/pods/16369e5b-22ee-4368-b364-37820c889fcc/volumes"
Sep 19 09:40:09.268950 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:09.268906    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:40:10.202224 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:10.202188    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:10.202720 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:10.202481    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:10.215067 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:10.215037    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:10.330436 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:10.330409    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:40:12.508341 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:12.508302    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:40:12.608776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:12.608742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:40:27.677455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.677421    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"]
Sep 19 09:40:27.677859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.677736    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16369e5b-22ee-4368-b364-37820c889fcc" containerName="main"
Sep 19 09:40:27.677859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.677746    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="16369e5b-22ee-4368-b364-37820c889fcc" containerName="main"
Sep 19 09:40:27.677859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.677761    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95df44b9-143b-4f44-8510-85739e72fb74" containerName="main"
Sep 19 09:40:27.677859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.677768    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="95df44b9-143b-4f44-8510-85739e72fb74" containerName="main"
Sep 19 09:40:27.677859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.677820    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="95df44b9-143b-4f44-8510-85739e72fb74" containerName="main"
Sep 19 09:40:27.677859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.677832    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="16369e5b-22ee-4368-b364-37820c889fcc" containerName="main"
Sep 19 09:40:27.682301 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.682281    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.685525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.685498    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"kube-root-ca.crt\""
Sep 19 09:40:27.685640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.685503    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"openshift-service-ca.crt\""
Sep 19 09:40:27.685640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.685534    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"default-dockercfg-g8624\""
Sep 19 09:40:27.685640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.685503    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"sub-test-model-1afa13-kserve-self-signed-certs\""
Sep 19 09:40:27.698512 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.698489    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"]
Sep 19 09:40:27.714332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.714306    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:40:27.763022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.762983    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-dshm\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.763212 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.763031    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6gm9\" (UniqueName: \"kubernetes.io/projected/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-kube-api-access-g6gm9\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.763212 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.763051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tmp-dir\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.763212 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.763171    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-home\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.763348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.763220    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-model-cache\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.763348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.763250    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tls-certs\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.863901 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.863858    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gm9\" (UniqueName: \"kubernetes.io/projected/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-kube-api-access-g6gm9\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.863901 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.863903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tmp-dir\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.864170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.863954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-home\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.864170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.864001    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-model-cache\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.864170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.864037    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tls-certs\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.864170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.864083    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-dshm\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.864444 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.864422    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-model-cache\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.864514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.864457    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tmp-dir\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.864514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.864465    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-home\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.866498 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.866477    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-dshm\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.866814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.866796    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tls-certs\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.872715 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.872693    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6gm9\" (UniqueName: \"kubernetes.io/projected/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-kube-api-access-g6gm9\") pod \"sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:27.991004 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:27.990912    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:28.271392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:28.271315    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"]
Sep 19 09:40:28.274530 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:40:28.274496    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod892c4b1a_a8fd_47a9_a533_6aaa79bbf206.slice/crio-1ebcdd21dc04fde7efc7e2105e459f06efa5f842d3c794f2bcf0d4526070f442 WatchSource:0}: Error finding container 1ebcdd21dc04fde7efc7e2105e459f06efa5f842d3c794f2bcf0d4526070f442: Status 404 returned error can't find the container with id 1ebcdd21dc04fde7efc7e2105e459f06efa5f842d3c794f2bcf0d4526070f442
Sep 19 09:40:28.403164 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:28.403129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5" event={"ID":"892c4b1a-a8fd-47a9-a533-6aaa79bbf206","Type":"ContainerStarted","Data":"687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47"}
Sep 19 09:40:28.403164 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:28.403169    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5" event={"ID":"892c4b1a-a8fd-47a9-a533-6aaa79bbf206","Type":"ContainerStarted","Data":"1ebcdd21dc04fde7efc7e2105e459f06efa5f842d3c794f2bcf0d4526070f442"}
Sep 19 09:40:28.426720 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:28.426668    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5" podStartSLOduration=1.426652348 podStartE2EDuration="1.426652348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:40:28.426342114 +0000 UTC m=+2042.113147413" watchObservedRunningTime="2026-09-19 09:40:28.426652348 +0000 UTC m=+2042.113457645"
Sep 19 09:40:31.400912 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:31.400876    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:40:37.991445 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:37.991402    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:37.991958 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:37.991464    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:38.004055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:38.004025    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:38.471260 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:38.471231    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:40:43.875399 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:43.875363    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:40:52.388994 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:52.388906    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:40:54.886704 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:54.886669    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7848-56c2c4"]
Sep 19 09:40:54.890395 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:54.890372    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:40:54.893055 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:54.893035    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-q5mhq\""
Sep 19 09:40:54.897048 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:54.897025    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7848-56c2c4"]
Sep 19 09:40:54.905315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:54.905269    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7f84\" (UniqueName: \"kubernetes.io/projected/b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5-kube-api-access-w7f84\") pod \"test-curl-7848-56c2c4\" (UID: \"b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5\") " pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:40:55.005998 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:55.005965    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7f84\" (UniqueName: \"kubernetes.io/projected/b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5-kube-api-access-w7f84\") pod \"test-curl-7848-56c2c4\" (UID: \"b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5\") " pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:40:55.019844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:55.019816    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7f84\" (UniqueName: \"kubernetes.io/projected/b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5-kube-api-access-w7f84\") pod \"test-curl-7848-56c2c4\" (UID: \"b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5\") " pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:40:55.202143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:55.202011    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:40:55.474842 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:55.474816    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7848-56c2c4"]
Sep 19 09:40:55.477602 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:40:55.477577    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb73a5b64_2c4c_473c_a7e8_e0adc0a35bc5.slice/crio-85130195a81d02acb1e293a7d17022a6db751c1ede6b5af894246bdb2d885edd WatchSource:0}: Error finding container 85130195a81d02acb1e293a7d17022a6db751c1ede6b5af894246bdb2d885edd: Status 404 returned error can't find the container with id 85130195a81d02acb1e293a7d17022a6db751c1ede6b5af894246bdb2d885edd
Sep 19 09:40:55.541250 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:55.541219    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7848-56c2c4" event={"ID":"b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5","Type":"ContainerStarted","Data":"85130195a81d02acb1e293a7d17022a6db751c1ede6b5af894246bdb2d885edd"}
Sep 19 09:40:56.547462 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:56.547426    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7848-56c2c4" event={"ID":"b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5","Type":"ContainerStarted","Data":"832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7"}
Sep 19 09:40:56.563523 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:56.563465    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7848-56c2c4" podStartSLOduration=2.563449125 podStartE2EDuration="2.563449125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:40:56.561602867 +0000 UTC m=+2070.248408166" watchObservedRunningTime="2026-09-19 09:40:56.563449125 +0000 UTC m=+2070.250254424"
Sep 19 09:40:57.157164 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.157125    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7848-56c2c4"]
Sep 19 09:40:57.162016 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.161982    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7848-56c2c4"]
Sep 19 09:40:57.276709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.276615    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7848-dafbbc"]
Sep 19 09:40:57.276938 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.276925    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" containerName="test-curl-7848-56c2c4"
Sep 19 09:40:57.276982 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.276940    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" containerName="test-curl-7848-56c2c4"
Sep 19 09:40:57.277028 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.277019    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" containerName="test-curl-7848-56c2c4"
Sep 19 09:40:57.280049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.280030    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:40:57.286801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.286771    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7848-dafbbc"]
Sep 19 09:40:57.326404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.326356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfjnf\" (UniqueName: \"kubernetes.io/projected/f7c17e6d-d3eb-4f55-b805-588c70873a93-kube-api-access-jfjnf\") pod \"test-curl-7848-dafbbc\" (UID: \"f7c17e6d-d3eb-4f55-b805-588c70873a93\") " pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:40:57.427197 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.427152    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfjnf\" (UniqueName: \"kubernetes.io/projected/f7c17e6d-d3eb-4f55-b805-588c70873a93-kube-api-access-jfjnf\") pod \"test-curl-7848-dafbbc\" (UID: \"f7c17e6d-d3eb-4f55-b805-588c70873a93\") " pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:40:57.435536 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.435506    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfjnf\" (UniqueName: \"kubernetes.io/projected/f7c17e6d-d3eb-4f55-b805-588c70873a93-kube-api-access-jfjnf\") pod \"test-curl-7848-dafbbc\" (UID: \"f7c17e6d-d3eb-4f55-b805-588c70873a93\") " pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:40:57.552643 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:40:57.552542    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7848-56c2c4" container="test-curl-7848-56c2c4"
Sep 19 09:40:57.590531 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.590490    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:40:57.860733 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:57.860667    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7848-dafbbc"]
Sep 19 09:40:57.863404 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:40:57.863369    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7c17e6d_d3eb_4f55_b805_588c70873a93.slice/crio-e732b56c788b42f63f9d8be26d7996cf560bd1a7d67eed041800822002bdec9a WatchSource:0}: Error finding container e732b56c788b42f63f9d8be26d7996cf560bd1a7d67eed041800822002bdec9a: Status 404 returned error can't find the container with id e732b56c788b42f63f9d8be26d7996cf560bd1a7d67eed041800822002bdec9a
Sep 19 09:40:58.559092 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:58.559054    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7848-dafbbc" event={"ID":"f7c17e6d-d3eb-4f55-b805-588c70873a93","Type":"ContainerStarted","Data":"aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38"}
Sep 19 09:40:58.559092 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:58.559114    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7848-dafbbc" event={"ID":"f7c17e6d-d3eb-4f55-b805-588c70873a93","Type":"ContainerStarted","Data":"e732b56c788b42f63f9d8be26d7996cf560bd1a7d67eed041800822002bdec9a"}
Sep 19 09:40:58.559566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:58.559235    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7848-56c2c4" podUID="b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" containerName="test-curl-7848-56c2c4" containerID="cri-o://832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7" gracePeriod=2
Sep 19 09:40:58.575877 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:58.575819    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7848-dafbbc" podStartSLOduration=1.575800001 podStartE2EDuration="1.575800001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:40:58.573930198 +0000 UTC m=+2072.260735497" watchObservedRunningTime="2026-09-19 09:40:58.575800001 +0000 UTC m=+2072.262605299"
Sep 19 09:40:58.576071 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:40:58.575961    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7848-56c2c4\" is forbidden: User \"system:node:ip-10-0-131-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-218.ec2.internal' and this object" podUID="b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:40:59.173898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:59.173862    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7848-dafbbc"]
Sep 19 09:40:59.184487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:59.184459    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7848-dafbbc"]
Sep 19 09:40:59.489470 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:40:59.489432    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:40:59.563429 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:40:59.563398    2581 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7848-dafbbc" container="test-curl-7848-dafbbc"
Sep 19 09:41:00.568905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:00.568874    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7848-dafbbc" podUID="f7c17e6d-d3eb-4f55-b805-588c70873a93" containerName="test-curl-7848-dafbbc" containerID="cri-o://aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38" gracePeriod=2
Sep 19 09:41:00.571061 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:41:00.571019    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7848-dafbbc\" is forbidden: User \"system:node:ip-10-0-131-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-218.ec2.internal' and this object" podUID="f7c17e6d-d3eb-4f55-b805-588c70873a93" pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:41:00.705749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:00.705720    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:41:00.708189 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:41:00.708166    2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7848-dafbbc\" is forbidden: User \"system:node:ip-10-0-131-218.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-218.ec2.internal' and this object" podUID="f7c17e6d-d3eb-4f55-b805-588c70873a93" pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:41:00.756355 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:00.756321    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7f84\" (UniqueName: \"kubernetes.io/projected/b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5-kube-api-access-w7f84\") pod \"b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5\" (UID: \"b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5\") "
Sep 19 09:41:00.758621 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:00.758595    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5-kube-api-access-w7f84" (OuterVolumeSpecName: "kube-api-access-w7f84") pod "b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" (UID: "b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5"). InnerVolumeSpecName "kube-api-access-w7f84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:41:00.830587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:00.830495    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" path="/var/lib/kubelet/pods/b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5/volumes"
Sep 19 09:41:00.858010 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:00.857977    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7f84\" (UniqueName: \"kubernetes.io/projected/b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5-kube-api-access-w7f84\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:01.211070 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:01.211033    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"]
Sep 19 09:41:01.244830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:01.244804    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:41:01.575026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:01.574930    2581 generic.go:356] "Generic (PLEG): container finished" podID="b73a5b64-2c4c-473c-a7e8-e0adc0a35bc5" containerID="832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7" exitCode=137
Sep 19 09:41:01.575026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:01.574992    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-56c2c4"
Sep 19 09:41:01.575518 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:01.575032    2581 scope.go:117] "RemoveContainer" containerID="832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7"
Sep 19 09:41:01.583740 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:01.583719    2581 scope.go:117] "RemoveContainer" containerID="832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7"
Sep 19 09:41:01.584031 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:41:01.584010    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7\": container with ID starting with 832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7 not found: ID does not exist" containerID="832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7"
Sep 19 09:41:01.584080 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:01.584041    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7"} err="failed to get container status \"832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7\": rpc error: code = NotFound desc = could not find container \"832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7\": container with ID starting with 832f970cb2485254c33a8c2c933dd55871168b7492a789ffb22933d1d088c0a7 not found: ID does not exist"
Sep 19 09:41:02.108905 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:02.108874    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"]
Sep 19 09:41:02.148671 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:02.148634    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:41:02.756085 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:02.756059    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:41:02.875863 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:02.875767    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfjnf\" (UniqueName: \"kubernetes.io/projected/f7c17e6d-d3eb-4f55-b805-588c70873a93-kube-api-access-jfjnf\") pod \"f7c17e6d-d3eb-4f55-b805-588c70873a93\" (UID: \"f7c17e6d-d3eb-4f55-b805-588c70873a93\") "
Sep 19 09:41:02.878026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:02.877992    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c17e6d-d3eb-4f55-b805-588c70873a93-kube-api-access-jfjnf" (OuterVolumeSpecName: "kube-api-access-jfjnf") pod "f7c17e6d-d3eb-4f55-b805-588c70873a93" (UID: "f7c17e6d-d3eb-4f55-b805-588c70873a93"). InnerVolumeSpecName "kube-api-access-jfjnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:41:02.977063 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:02.977024    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfjnf\" (UniqueName: \"kubernetes.io/projected/f7c17e6d-d3eb-4f55-b805-588c70873a93-kube-api-access-jfjnf\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:03.585699 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:03.585663    2581 generic.go:356] "Generic (PLEG): container finished" podID="f7c17e6d-d3eb-4f55-b805-588c70873a93" containerID="aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38" exitCode=137
Sep 19 09:41:03.585873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:03.585711    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7848-dafbbc"
Sep 19 09:41:03.585873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:03.585768    2581 scope.go:117] "RemoveContainer" containerID="aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38"
Sep 19 09:41:03.594430 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:03.594410    2581 scope.go:117] "RemoveContainer" containerID="aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38"
Sep 19 09:41:03.594720 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:41:03.594701    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38\": container with ID starting with aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38 not found: ID does not exist" containerID="aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38"
Sep 19 09:41:03.594785 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:03.594732    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38"} err="failed to get container status \"aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38\": rpc error: code = NotFound desc = could not find container \"aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38\": container with ID starting with aa6be18de4d39cffab6e28c96a955dc5acd788e3431395f3f3a4b844aacb1e38 not found: ID does not exist"
Sep 19 09:41:04.831029 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:04.830984    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c17e6d-d3eb-4f55-b805-588c70873a93" path="/var/lib/kubelet/pods/f7c17e6d-d3eb-4f55-b805-588c70873a93/volumes"
Sep 19 09:41:07.677521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:07.677476    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:41:16.230469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.230392    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4" podUID="7c84f7d3-6f41-4de1-9e4c-6f28f7878848" containerName="main" containerID="cri-o://7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986" gracePeriod=45
Sep 19 09:41:16.478795 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.478770    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:41:16.575232 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.575154    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"]
Sep 19 09:41:16.575501 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.575489    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c17e6d-d3eb-4f55-b805-588c70873a93" containerName="test-curl-7848-dafbbc"
Sep 19 09:41:16.575548 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.575503    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c17e6d-d3eb-4f55-b805-588c70873a93" containerName="test-curl-7848-dafbbc"
Sep 19 09:41:16.575548 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.575514    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c84f7d3-6f41-4de1-9e4c-6f28f7878848" containerName="main"
Sep 19 09:41:16.575548 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.575520    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c84f7d3-6f41-4de1-9e4c-6f28f7878848" containerName="main"
Sep 19 09:41:16.575645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.575570    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c17e6d-d3eb-4f55-b805-588c70873a93" containerName="test-curl-7848-dafbbc"
Sep 19 09:41:16.575645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.575581    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c84f7d3-6f41-4de1-9e4c-6f28f7878848" containerName="main"
Sep 19 09:41:16.578347 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.578330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.580701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.580668    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-3240ec\"/\"rate-test-model-3240ec-kserve-self-signed-certs\""
Sep 19 09:41:16.589058 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.589035    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"]
Sep 19 09:41:16.595570 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595546    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztnhm\" (UniqueName: \"kubernetes.io/projected/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-kube-api-access-ztnhm\") pod \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") "
Sep 19 09:41:16.595683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595576    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-model-cache\") pod \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") "
Sep 19 09:41:16.595683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595619    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tls-certs\") pod \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") "
Sep 19 09:41:16.595683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595649    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-home\") pod \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") "
Sep 19 09:41:16.595683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595665    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tmp-dir\") pod \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") "
Sep 19 09:41:16.595903 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-dshm\") pod \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\" (UID: \"7c84f7d3-6f41-4de1-9e4c-6f28f7878848\") "
Sep 19 09:41:16.595964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595917    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-model-cache" (OuterVolumeSpecName: "model-cache") pod "7c84f7d3-6f41-4de1-9e4c-6f28f7878848" (UID: "7c84f7d3-6f41-4de1-9e4c-6f28f7878848"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:16.595964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.595941    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-home" (OuterVolumeSpecName: "home") pod "7c84f7d3-6f41-4de1-9e4c-6f28f7878848" (UID: "7c84f7d3-6f41-4de1-9e4c-6f28f7878848"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:16.596374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.596340    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7c84f7d3-6f41-4de1-9e4c-6f28f7878848" (UID: "7c84f7d3-6f41-4de1-9e4c-6f28f7878848"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:16.598149 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.598124    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-dshm" (OuterVolumeSpecName: "dshm") pod "7c84f7d3-6f41-4de1-9e4c-6f28f7878848" (UID: "7c84f7d3-6f41-4de1-9e4c-6f28f7878848"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:16.598659 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.598632    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7c84f7d3-6f41-4de1-9e4c-6f28f7878848" (UID: "7c84f7d3-6f41-4de1-9e4c-6f28f7878848"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:41:16.598659 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.598645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-kube-api-access-ztnhm" (OuterVolumeSpecName: "kube-api-access-ztnhm") pod "7c84f7d3-6f41-4de1-9e4c-6f28f7878848" (UID: "7c84f7d3-6f41-4de1-9e4c-6f28f7878848"). InnerVolumeSpecName "kube-api-access-ztnhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:41:16.615002 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.614975    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:41:16.649519 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.649481    2581 generic.go:356] "Generic (PLEG): container finished" podID="7c84f7d3-6f41-4de1-9e4c-6f28f7878848" containerID="7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986" exitCode=0
Sep 19 09:41:16.649658 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.649564    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"
Sep 19 09:41:16.649658 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.649575    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4" event={"ID":"7c84f7d3-6f41-4de1-9e4c-6f28f7878848","Type":"ContainerDied","Data":"7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986"}
Sep 19 09:41:16.649658 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.649627    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4" event={"ID":"7c84f7d3-6f41-4de1-9e4c-6f28f7878848","Type":"ContainerDied","Data":"afc310769d9c89df0c6d40b23062c51ba8d4ef0f01dc2a42c457db7216606769"}
Sep 19 09:41:16.649658 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.649652    2581 scope.go:117] "RemoveContainer" containerID="7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986"
Sep 19 09:41:16.658997 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.658980    2581 scope.go:117] "RemoveContainer" containerID="7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986"
Sep 19 09:41:16.659321 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:41:16.659298    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986\": container with ID starting with 7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986 not found: ID does not exist" containerID="7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986"
Sep 19 09:41:16.659412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.659326    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986"} err="failed to get container status \"7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986\": rpc error: code = NotFound desc = could not find container \"7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986\": container with ID starting with 7d97a93ea3f749eae9474765ccf794df0c3d3f08b5a9ec28e6e0268b2c938986 not found: ID does not exist"
Sep 19 09:41:16.671018 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.670994    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"]
Sep 19 09:41:16.672630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.672605    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/sub-test-model-3240ec-kserve-7b648fd4b5-zzcb4"]
Sep 19 09:41:16.697198 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tmp-dir\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.697348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697212    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tls-certs\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.697348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697242    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-home\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.697348 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697308    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l79bg\" (UniqueName: \"kubernetes.io/projected/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-kube-api-access-l79bg\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.697494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-dshm\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.697494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-model-cache\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.697494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697469    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztnhm\" (UniqueName: \"kubernetes.io/projected/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-kube-api-access-ztnhm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:16.697494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697481    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:16.697494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697490    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:16.697494 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697497    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:16.697679 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697507    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:16.697679 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.697528    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c84f7d3-6f41-4de1-9e4c-6f28f7878848-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:16.798203 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798164    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tmp-dir\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798212    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tls-certs\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-home\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798281    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l79bg\" (UniqueName: \"kubernetes.io/projected/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-kube-api-access-l79bg\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798318    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-dshm\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798415 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798345    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-model-cache\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798616    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tmp-dir\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798624    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-home\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.798701 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.798692    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-model-cache\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.800747 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.800725    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-dshm\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.801003 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.800987    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tls-certs\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.807223 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.807205    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l79bg\" (UniqueName: \"kubernetes.io/projected/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-kube-api-access-l79bg\") pod \"rate-test-model-3240ec-kserve-79ccf668bc-c8zwt\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") " pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:16.830856 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.830780    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c84f7d3-6f41-4de1-9e4c-6f28f7878848" path="/var/lib/kubelet/pods/7c84f7d3-6f41-4de1-9e4c-6f28f7878848/volumes"
Sep 19 09:41:16.888246 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:16.888212    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:17.128353 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.128291    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5" podUID="892c4b1a-a8fd-47a9-a533-6aaa79bbf206" containerName="main" containerID="cri-o://687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47" gracePeriod=45
Sep 19 09:41:17.165460 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.165426    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"]
Sep 19 09:41:17.181046 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:41:17.181017    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf7fb4a7_9dbf_4007_be6c_4870ca28b3a1.slice/crio-6df1fb7fb2792624fdadfaf7fafb8b1db73c4328360d183b9413305e5853a117 WatchSource:0}: Error finding container 6df1fb7fb2792624fdadfaf7fafb8b1db73c4328360d183b9413305e5853a117: Status 404 returned error can't find the container with id 6df1fb7fb2792624fdadfaf7fafb8b1db73c4328360d183b9413305e5853a117
Sep 19 09:41:17.384875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.384813    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:41:17.505642 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505595    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g6gm9\" (UniqueName: \"kubernetes.io/projected/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-kube-api-access-g6gm9\") pod \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") "
Sep 19 09:41:17.505847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505651    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-home\") pod \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") "
Sep 19 09:41:17.505847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505698    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-model-cache\") pod \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") "
Sep 19 09:41:17.505847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505752    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tls-certs\") pod \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") "
Sep 19 09:41:17.505847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505789    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-dshm\") pod \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") "
Sep 19 09:41:17.505847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505831    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tmp-dir\") pod \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\" (UID: \"892c4b1a-a8fd-47a9-a533-6aaa79bbf206\") "
Sep 19 09:41:17.506143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505974    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-home" (OuterVolumeSpecName: "home") pod "892c4b1a-a8fd-47a9-a533-6aaa79bbf206" (UID: "892c4b1a-a8fd-47a9-a533-6aaa79bbf206"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:17.506143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.505996    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-model-cache" (OuterVolumeSpecName: "model-cache") pod "892c4b1a-a8fd-47a9-a533-6aaa79bbf206" (UID: "892c4b1a-a8fd-47a9-a533-6aaa79bbf206"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:17.506143 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.506094    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:17.506304 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.506148    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:17.506304 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.506192    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "892c4b1a-a8fd-47a9-a533-6aaa79bbf206" (UID: "892c4b1a-a8fd-47a9-a533-6aaa79bbf206"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:17.508039 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.508015    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-kube-api-access-g6gm9" (OuterVolumeSpecName: "kube-api-access-g6gm9") pod "892c4b1a-a8fd-47a9-a533-6aaa79bbf206" (UID: "892c4b1a-a8fd-47a9-a533-6aaa79bbf206"). InnerVolumeSpecName "kube-api-access-g6gm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:41:17.508159 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.508056    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-dshm" (OuterVolumeSpecName: "dshm") pod "892c4b1a-a8fd-47a9-a533-6aaa79bbf206" (UID: "892c4b1a-a8fd-47a9-a533-6aaa79bbf206"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:41:17.508159 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.508128    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "892c4b1a-a8fd-47a9-a533-6aaa79bbf206" (UID: "892c4b1a-a8fd-47a9-a533-6aaa79bbf206"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:41:17.607404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.607363    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:17.607404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.607394    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:17.607404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.607406    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6gm9\" (UniqueName: \"kubernetes.io/projected/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-kube-api-access-g6gm9\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:17.607404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.607415    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/892c4b1a-a8fd-47a9-a533-6aaa79bbf206-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:41:17.656302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.656215    2581 generic.go:356] "Generic (PLEG): container finished" podID="892c4b1a-a8fd-47a9-a533-6aaa79bbf206" containerID="687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47" exitCode=0
Sep 19 09:41:17.656302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.656255    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"
Sep 19 09:41:17.656523 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.656313    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5" event={"ID":"892c4b1a-a8fd-47a9-a533-6aaa79bbf206","Type":"ContainerDied","Data":"687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47"}
Sep 19 09:41:17.656523 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.656345    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5" event={"ID":"892c4b1a-a8fd-47a9-a533-6aaa79bbf206","Type":"ContainerDied","Data":"1ebcdd21dc04fde7efc7e2105e459f06efa5f842d3c794f2bcf0d4526070f442"}
Sep 19 09:41:17.656523 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.656362    2581 scope.go:117] "RemoveContainer" containerID="687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47"
Sep 19 09:41:17.658138 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.658067    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt" event={"ID":"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1","Type":"ContainerStarted","Data":"cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920"}
Sep 19 09:41:17.658138 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.658130    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt" event={"ID":"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1","Type":"ContainerStarted","Data":"6df1fb7fb2792624fdadfaf7fafb8b1db73c4328360d183b9413305e5853a117"}
Sep 19 09:41:17.665225 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.665208    2581 scope.go:117] "RemoveContainer" containerID="687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47"
Sep 19 09:41:17.665492 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:41:17.665471    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47\": container with ID starting with 687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47 not found: ID does not exist" containerID="687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47"
Sep 19 09:41:17.665561 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.665500    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47"} err="failed to get container status \"687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47\": rpc error: code = NotFound desc = could not find container \"687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47\": container with ID starting with 687bd03b17bef0fa4cb7d5ee229ee57dd3e7c1a54f53ccd0e2ae70c5ecb23b47 not found: ID does not exist"
Sep 19 09:41:17.677146 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.677088    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt" podStartSLOduration=1.677073172 podStartE2EDuration="1.677073172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:41:17.674455245 +0000 UTC m=+2091.361260544" watchObservedRunningTime="2026-09-19 09:41:17.677073172 +0000 UTC m=+2091.363878469"
Sep 19 09:41:17.687749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.687727    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"]
Sep 19 09:41:17.691478 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:17.691457    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/sub-test-model-1afa13-kserve-77bb74cbf7-gxjb5"]
Sep 19 09:41:18.830471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:18.830440    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="892c4b1a-a8fd-47a9-a533-6aaa79bbf206" path="/var/lib/kubelet/pods/892c4b1a-a8fd-47a9-a533-6aaa79bbf206/volumes"
Sep 19 09:41:19.783313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:19.783277    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:41:20.729879 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:20.729844    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:41:20.831626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:20.831595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:41:26.888502 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:26.888444    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:26.888986 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:26.888515    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:26.901663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:26.901636    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:27.717891 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:27.717861    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:41:41.227044 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.227007    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"]
Sep 19 09:41:41.227440 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.227338    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="892c4b1a-a8fd-47a9-a533-6aaa79bbf206" containerName="main"
Sep 19 09:41:41.227440 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.227350    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="892c4b1a-a8fd-47a9-a533-6aaa79bbf206" containerName="main"
Sep 19 09:41:41.227440 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.227400    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="892c4b1a-a8fd-47a9-a533-6aaa79bbf206" containerName="main"
Sep 19 09:41:41.230320 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.230304    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.233781 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.233745    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"kube-root-ca.crt\""
Sep 19 09:41:41.235015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.234988    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"default-dockercfg-g8624\""
Sep 19 09:41:41.235281 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.235262    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"openshift-service-ca.crt\""
Sep 19 09:41:41.235541 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.235522    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1afa13\"/\"rate-test-model-1afa13-kserve-self-signed-certs\""
Sep 19 09:41:41.253534 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.253500    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"]
Sep 19 09:41:41.286550 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.286506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:41:41.313866 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.313833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-tmp-dir\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.313866 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.313871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adf2416c-e7ac-4a18-82c6-0f3f81632738-tls-certs\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.314079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.313894    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-dshm\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.314079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.313909    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-model-cache\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.314079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.313993    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-home\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.314079 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.314023    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f25h\" (UniqueName: \"kubernetes.io/projected/adf2416c-e7ac-4a18-82c6-0f3f81632738-kube-api-access-7f25h\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415074 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415033    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-tmp-dir\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415080    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adf2416c-e7ac-4a18-82c6-0f3f81632738-tls-certs\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415143    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-dshm\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415166    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-model-cache\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415196    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-home\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415229    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f25h\" (UniqueName: \"kubernetes.io/projected/adf2416c-e7ac-4a18-82c6-0f3f81632738-kube-api-access-7f25h\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415579 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415532    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-tmp-dir\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415638 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415618    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-home\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.415677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.415664    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-model-cache\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.417749 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.417724    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-dshm\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.417920 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.417904    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adf2416c-e7ac-4a18-82c6-0f3f81632738-tls-certs\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.424437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.424400    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f25h\" (UniqueName: \"kubernetes.io/projected/adf2416c-e7ac-4a18-82c6-0f3f81632738-kube-api-access-7f25h\") pod \"rate-test-model-1afa13-kserve-5578c76cbc-6jgtd\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") " pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.544914 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.544823    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:41.820216 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:41.820190    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"]
Sep 19 09:41:41.821801 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:41:41.821773    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadf2416c_e7ac_4a18_82c6_0f3f81632738.slice/crio-e8fb0acb9c4804b67fd8f5ee48e723d79cc2635e59a3f72bb73c6a151bc5fb42 WatchSource:0}: Error finding container e8fb0acb9c4804b67fd8f5ee48e723d79cc2635e59a3f72bb73c6a151bc5fb42: Status 404 returned error can't find the container with id e8fb0acb9c4804b67fd8f5ee48e723d79cc2635e59a3f72bb73c6a151bc5fb42
Sep 19 09:41:42.782814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:42.782768    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd" event={"ID":"adf2416c-e7ac-4a18-82c6-0f3f81632738","Type":"ContainerStarted","Data":"a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c"}
Sep 19 09:41:42.782814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:42.782816    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd" event={"ID":"adf2416c-e7ac-4a18-82c6-0f3f81632738","Type":"ContainerStarted","Data":"e8fb0acb9c4804b67fd8f5ee48e723d79cc2635e59a3f72bb73c6a151bc5fb42"}
Sep 19 09:41:42.801490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:42.801277    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd" podStartSLOduration=1.801257527 podStartE2EDuration="1.801257527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:41:42.800837087 +0000 UTC m=+2116.487642387" watchObservedRunningTime="2026-09-19 09:41:42.801257527 +0000 UTC m=+2116.488062827"
Sep 19 09:41:44.782308 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:44.782271    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:41:51.545296 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:51.545239    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:51.545804 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:51.545315    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:51.558082 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:51.558056    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:51.836375 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:51.836287    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:41:54.526308 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:41:54.526275    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:01.751597 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:01.751559    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:10.572757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:10.572708    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:42:21.277970 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:21.277926    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:42:28.206835 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:28.206800    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"]
Sep 19 09:42:28.219008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:28.218980    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:29.248790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:29.248754    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"]
Sep 19 09:42:29.276137 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:29.276086    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:29.693686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:29.693648    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:29.997624 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:29.997523    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:30.367549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:30.367462    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:32.105276 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:32.105235    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:32.471349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:32.471315    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:42:32.795473 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:32.795388    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:32.891561 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:32.891529    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:34.065910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:34.065873    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:34.994089 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:34.994059    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"]
Sep 19 09:42:34.997452 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:34.997423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.000993 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.000967    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzltl\""
Sep 19 09:42:35.001234 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.001040    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 09:42:35.001234 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.001046    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-6fd1a9-serving-cert\""
Sep 19 09:42:35.001897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.001876    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:42:35.002011 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.001955    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 09:42:35.010475 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.010448    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"]
Sep 19 09:42:35.065257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.065222    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:35.099115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.099076    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dp9x\" (UniqueName: \"kubernetes.io/projected/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-kube-api-access-5dp9x\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.099495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.099146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-metrics-certs\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.099495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.099181    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.099495 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.099251    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.168683 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.168648    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:35.199924 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.199887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dp9x\" (UniqueName: \"kubernetes.io/projected/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-kube-api-access-5dp9x\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.199924 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.199928    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-metrics-certs\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.200205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.199951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.200205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.200158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.200316 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:42:35.200295    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-serving-cert: secret "maas-api-e2e-ipp-a-6fd1a9-serving-cert" not found
Sep 19 09:42:35.200426 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:42:35.200412    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls podName:cfbaced6-9743-4b4d-b5bb-1c810ddd91a7 nodeName:}" failed. No retries permitted until 2026-09-19 09:42:35.700387985 +0000 UTC m=+2169.387193275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls") pod "maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" (UID: "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7") : secret "maas-api-e2e-ipp-a-6fd1a9-serving-cert" not found
Sep 19 09:42:35.200648 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.200628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.202706 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.202684    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-metrics-certs\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.214261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.214236    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dp9x\" (UniqueName: \"kubernetes.io/projected/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-kube-api-access-5dp9x\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.621983 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.621945    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:35.703861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.703820    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.706551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.706518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:35.889392 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.889357    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:35.910784 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:35.910745    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:36.005543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.005503    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:36.129374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.129330    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"]
Sep 19 09:42:36.133033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.133009    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.135426 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.135407    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-6fd1a9-dockercfg-dqpzg\""
Sep 19 09:42:36.135979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.135962    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-6fd1a9\""
Sep 19 09:42:36.146795 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.146769    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"]
Sep 19 09:42:36.190994 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.190968    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"]
Sep 19 09:42:36.193531 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:42:36.193509    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfbaced6_9743_4b4d_b5bb_1c810ddd91a7.slice/crio-1679ce6655ade566770591fb3ae0a5c7253d2ef991be3d22f544d889ec8c071f WatchSource:0}: Error finding container 1679ce6655ade566770591fb3ae0a5c7253d2ef991be3d22f544d889ec8c071f: Status 404 returned error can't find the container with id 1679ce6655ade566770591fb3ae0a5c7253d2ef991be3d22f544d889ec8c071f
Sep 19 09:42:36.207384 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.207352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghm2r\" (UniqueName: \"kubernetes.io/projected/ff78d409-13d1-4c4c-b939-962ae9d46d63-kube-api-access-ghm2r\") pod \"payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.207525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.207415    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff78d409-13d1-4c4c-b939-962ae9d46d63-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.308771 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.308734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff78d409-13d1-4c4c-b939-962ae9d46d63-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.308957 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.308818    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghm2r\" (UniqueName: \"kubernetes.io/projected/ff78d409-13d1-4c4c-b939-962ae9d46d63-kube-api-access-ghm2r\") pod \"payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.309514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.309493    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff78d409-13d1-4c4c-b939-962ae9d46d63-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.319666 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.319635    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghm2r\" (UniqueName: \"kubernetes.io/projected/ff78d409-13d1-4c4c-b939-962ae9d46d63-kube-api-access-ghm2r\") pod \"payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.443297 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.443207    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:36.725298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:36.725274    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"]
Sep 19 09:42:36.727543 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:42:36.727515    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff78d409_13d1_4c4c_b939_962ae9d46d63.slice/crio-7620969dbe06f227c4abf94989381abff0506fc71d6ef2d31c368f5462935ae6 WatchSource:0}: Error finding container 7620969dbe06f227c4abf94989381abff0506fc71d6ef2d31c368f5462935ae6: Status 404 returned error can't find the container with id 7620969dbe06f227c4abf94989381abff0506fc71d6ef2d31c368f5462935ae6
Sep 19 09:42:37.038956 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.038849    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" event={"ID":"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7","Type":"ContainerStarted","Data":"fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04"}
Sep 19 09:42:37.038956 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.038915    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" event={"ID":"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7","Type":"ContainerStarted","Data":"1679ce6655ade566770591fb3ae0a5c7253d2ef991be3d22f544d889ec8c071f"}
Sep 19 09:42:37.039228 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.039026    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:37.040549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.040520    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb" event={"ID":"ff78d409-13d1-4c4c-b939-962ae9d46d63","Type":"ContainerStarted","Data":"360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d"}
Sep 19 09:42:37.040677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.040571    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb" event={"ID":"ff78d409-13d1-4c4c-b939-962ae9d46d63","Type":"ContainerStarted","Data":"7620969dbe06f227c4abf94989381abff0506fc71d6ef2d31c368f5462935ae6"}
Sep 19 09:42:37.040677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.040600    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:37.057286 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.057236    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" podStartSLOduration=2.670333835 podStartE2EDuration="3.057219188s" totalImagesPullingTime="386.885353ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:42:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:42:37.056657411 +0000 UTC m=+2170.743462712" watchObservedRunningTime="2026-09-19 09:42:37.057219188 +0000 UTC m=+2170.744024485"
Sep 19 09:42:37.077944 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:37.077893    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb" podStartSLOduration=1.077879342 podStartE2EDuration="1.077879342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:42:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:42:37.075489752 +0000 UTC m=+2170.762295049" watchObservedRunningTime="2026-09-19 09:42:37.077879342 +0000 UTC m=+2170.764684640"
Sep 19 09:42:43.051587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.051554    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:42:43.226477 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.226432    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt" podUID="df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" containerName="main" containerID="cri-o://cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920" gracePeriod=45
Sep 19 09:42:43.465167 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.465136    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:42:43.573297 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573264    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-dshm\") pod \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") "
Sep 19 09:42:43.573297 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573312    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tmp-dir\") pod \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") "
Sep 19 09:42:43.573520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573356    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tls-certs\") pod \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") "
Sep 19 09:42:43.573520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573376    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l79bg\" (UniqueName: \"kubernetes.io/projected/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-kube-api-access-l79bg\") pod \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") "
Sep 19 09:42:43.573520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573441    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-home\") pod \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") "
Sep 19 09:42:43.573520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573462    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-model-cache\") pod \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\" (UID: \"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1\") "
Sep 19 09:42:43.573725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573645    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" (UID: "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:43.573805 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573757    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-home" (OuterVolumeSpecName: "home") pod "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" (UID: "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:43.573960 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.573828    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-model-cache" (OuterVolumeSpecName: "model-cache") pod "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" (UID: "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:43.575733 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.575700    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-dshm" (OuterVolumeSpecName: "dshm") pod "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" (UID: "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:43.575836 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.575768    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" (UID: "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:42:43.575836 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.575779    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-kube-api-access-l79bg" (OuterVolumeSpecName: "kube-api-access-l79bg") pod "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" (UID: "df7fb4a7-9dbf-4007-be6c-4870ca28b3a1"). InnerVolumeSpecName "kube-api-access-l79bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:42:43.674617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.674577    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:43.674617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.674611    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:43.674617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.674622    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:43.674858 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.674631    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l79bg\" (UniqueName: \"kubernetes.io/projected/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-kube-api-access-l79bg\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:43.674858 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.674640    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:43.674858 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:43.674648    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:44.076471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.076386    2581 generic.go:356] "Generic (PLEG): container finished" podID="df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" containerID="cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920" exitCode=0
Sep 19 09:42:44.076471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.076429    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"
Sep 19 09:42:44.076926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.076472    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt" event={"ID":"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1","Type":"ContainerDied","Data":"cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920"}
Sep 19 09:42:44.076926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.076512    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt" event={"ID":"df7fb4a7-9dbf-4007-be6c-4870ca28b3a1","Type":"ContainerDied","Data":"6df1fb7fb2792624fdadfaf7fafb8b1db73c4328360d183b9413305e5853a117"}
Sep 19 09:42:44.076926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.076529    2581 scope.go:117] "RemoveContainer" containerID="cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920"
Sep 19 09:42:44.085879 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.085863    2581 scope.go:117] "RemoveContainer" containerID="cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920"
Sep 19 09:42:44.086150 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:42:44.086134    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920\": container with ID starting with cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920 not found: ID does not exist" containerID="cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920"
Sep 19 09:42:44.086195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.086158    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920"} err="failed to get container status \"cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920\": rpc error: code = NotFound desc = could not find container \"cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920\": container with ID starting with cfbc4c26ab85e0f86a4b8e14a0772affd6174cf8a63ea532b6d31b47fa1ec920 not found: ID does not exist"
Sep 19 09:42:44.098436 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.098412    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"]
Sep 19 09:42:44.101933 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.101911    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-3240ec/rate-test-model-3240ec-kserve-79ccf668bc-c8zwt"]
Sep 19 09:42:44.268161 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.268083    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd" podUID="adf2416c-e7ac-4a18-82c6-0f3f81632738" containerName="main" containerID="cri-o://a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c" gracePeriod=45
Sep 19 09:42:44.524057 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.524032    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:42:44.582708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.582674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-dshm\") pod \"adf2416c-e7ac-4a18-82c6-0f3f81632738\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") "
Sep 19 09:42:44.582910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.582716    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-model-cache\") pod \"adf2416c-e7ac-4a18-82c6-0f3f81632738\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") "
Sep 19 09:42:44.582910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.582781    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-tmp-dir\") pod \"adf2416c-e7ac-4a18-82c6-0f3f81632738\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") "
Sep 19 09:42:44.582910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.582813    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f25h\" (UniqueName: \"kubernetes.io/projected/adf2416c-e7ac-4a18-82c6-0f3f81632738-kube-api-access-7f25h\") pod \"adf2416c-e7ac-4a18-82c6-0f3f81632738\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") "
Sep 19 09:42:44.582910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.582851    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adf2416c-e7ac-4a18-82c6-0f3f81632738-tls-certs\") pod \"adf2416c-e7ac-4a18-82c6-0f3f81632738\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") "
Sep 19 09:42:44.582910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.582880    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-home\") pod \"adf2416c-e7ac-4a18-82c6-0f3f81632738\" (UID: \"adf2416c-e7ac-4a18-82c6-0f3f81632738\") "
Sep 19 09:42:44.583200 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.583090    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "adf2416c-e7ac-4a18-82c6-0f3f81632738" (UID: "adf2416c-e7ac-4a18-82c6-0f3f81632738"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:44.583200 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.583128    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-model-cache" (OuterVolumeSpecName: "model-cache") pod "adf2416c-e7ac-4a18-82c6-0f3f81632738" (UID: "adf2416c-e7ac-4a18-82c6-0f3f81632738"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:44.583315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.583276    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-home" (OuterVolumeSpecName: "home") pod "adf2416c-e7ac-4a18-82c6-0f3f81632738" (UID: "adf2416c-e7ac-4a18-82c6-0f3f81632738"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:44.585187 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.585151    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/adf2416c-e7ac-4a18-82c6-0f3f81632738-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "adf2416c-e7ac-4a18-82c6-0f3f81632738" (UID: "adf2416c-e7ac-4a18-82c6-0f3f81632738"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:42:44.585313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.585248    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-dshm" (OuterVolumeSpecName: "dshm") pod "adf2416c-e7ac-4a18-82c6-0f3f81632738" (UID: "adf2416c-e7ac-4a18-82c6-0f3f81632738"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:42:44.585313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.585260    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf2416c-e7ac-4a18-82c6-0f3f81632738-kube-api-access-7f25h" (OuterVolumeSpecName: "kube-api-access-7f25h") pod "adf2416c-e7ac-4a18-82c6-0f3f81632738" (UID: "adf2416c-e7ac-4a18-82c6-0f3f81632738"). InnerVolumeSpecName "kube-api-access-7f25h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:42:44.684205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.684088    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adf2416c-e7ac-4a18-82c6-0f3f81632738-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:44.684205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.684150    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:44.684205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.684158    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:44.684205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.684166    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:44.684205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.684173    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/adf2416c-e7ac-4a18-82c6-0f3f81632738-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:44.684205 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.684200    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f25h\" (UniqueName: \"kubernetes.io/projected/adf2416c-e7ac-4a18-82c6-0f3f81632738-kube-api-access-7f25h\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:42:44.832215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:44.832180    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" path="/var/lib/kubelet/pods/df7fb4a7-9dbf-4007-be6c-4870ca28b3a1/volumes"
Sep 19 09:42:45.084694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.084611    2581 generic.go:356] "Generic (PLEG): container finished" podID="adf2416c-e7ac-4a18-82c6-0f3f81632738" containerID="a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c" exitCode=0
Sep 19 09:42:45.084694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.084657    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"
Sep 19 09:42:45.085237 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.084692    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd" event={"ID":"adf2416c-e7ac-4a18-82c6-0f3f81632738","Type":"ContainerDied","Data":"a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c"}
Sep 19 09:42:45.085237 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.084733    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd" event={"ID":"adf2416c-e7ac-4a18-82c6-0f3f81632738","Type":"ContainerDied","Data":"e8fb0acb9c4804b67fd8f5ee48e723d79cc2635e59a3f72bb73c6a151bc5fb42"}
Sep 19 09:42:45.085237 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.084755    2581 scope.go:117] "RemoveContainer" containerID="a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c"
Sep 19 09:42:45.093387 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.093368    2581 scope.go:117] "RemoveContainer" containerID="a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c"
Sep 19 09:42:45.093684 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:42:45.093665    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c\": container with ID starting with a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c not found: ID does not exist" containerID="a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c"
Sep 19 09:42:45.093782 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.093691    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c"} err="failed to get container status \"a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c\": rpc error: code = NotFound desc = could not find container \"a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c\": container with ID starting with a1f8a2280a6b16ebe9314372ef8eed91756658f021751cc08c9b06b473a7406c not found: ID does not exist"
Sep 19 09:42:45.102180 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.102157    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"]
Sep 19 09:42:45.106056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:45.106030    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1afa13/rate-test-model-1afa13-kserve-5578c76cbc-6jgtd"]
Sep 19 09:42:46.775364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:46.775326    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:42:46.831766 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:46.831724    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adf2416c-e7ac-4a18-82c6-0f3f81632738" path="/var/lib/kubelet/pods/adf2416c-e7ac-4a18-82c6-0f3f81632738/volumes"
Sep 19 09:42:48.048568 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:48.048535    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:42:57.620469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:57.620350    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:57.752172 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:57.752134    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:58.318852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.318814    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:58.442458 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.442417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:58.739078 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.739044    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"]
Sep 19 09:42:58.739490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.739390    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adf2416c-e7ac-4a18-82c6-0f3f81632738" containerName="main"
Sep 19 09:42:58.739490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.739402    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="adf2416c-e7ac-4a18-82c6-0f3f81632738" containerName="main"
Sep 19 09:42:58.739490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.739414    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" containerName="main"
Sep 19 09:42:58.739490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.739419    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" containerName="main"
Sep 19 09:42:58.739490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.739469    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="adf2416c-e7ac-4a18-82c6-0f3f81632738" containerName="main"
Sep 19 09:42:58.739490 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.739477    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="df7fb4a7-9dbf-4007-be6c-4870ca28b3a1" containerName="main"
Sep 19 09:42:58.742486 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.742462    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.745088 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.745062    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 09:42:58.745287 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.745144    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-e10409-openshift-default-dockercfg-rwq2j\""
Sep 19 09:42:58.760141 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.760088    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"]
Sep 19 09:42:58.792315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.792267    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:58.803332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803302    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-socket\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803350    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbm7\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-kube-api-access-8rbm7\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803463 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803417    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istiod-ca-cert\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803545 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-credential-socket\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803545 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-envoy\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803545 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-podinfo\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803646 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803555    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-token\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803646 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803584    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-data\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.803646 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.803607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-certs\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.893817 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.893776    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:58.904465 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904430    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-certs\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.904641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904477    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-socket\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.904641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbm7\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-kube-api-access-8rbm7\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.904641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istiod-ca-cert\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.904641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904560    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-credential-socket\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.904641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904581    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-envoy\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.904641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904606    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-podinfo\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.904641 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-token\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.905022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904670    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-data\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.905022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904870    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-certs\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.905022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904947    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-socket\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.905022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.904992    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-credential-socket\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.905208 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.905037    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-data\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.905371 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.905348    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istiod-ca-cert\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.907135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.907116    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-envoy\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.907409 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.907389    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-podinfo\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.913320 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.913301    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-token\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:58.913958 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:58.913939    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbm7\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-kube-api-access-8rbm7\") pod \"e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") " pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:59.053163 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:59.053046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:42:59.123876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:59.123816    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:42:59.350235 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:59.350204    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"]
Sep 19 09:42:59.350857 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:42:59.350832    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d0f487_e80f_42d7_bfdf_ce63c922bec4.slice/crio-09791c5600b377b9c16cdb0e46a6788c46f95e31e1a732cb5b29837b8df86b77 WatchSource:0}: Error finding container 09791c5600b377b9c16cdb0e46a6788c46f95e31e1a732cb5b29837b8df86b77: Status 404 returned error can't find the container with id 09791c5600b377b9c16cdb0e46a6788c46f95e31e1a732cb5b29837b8df86b77
Sep 19 09:42:59.353261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:59.353226    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:42:59.353331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:59.353295    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:42:59.353331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:42:59.353323    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:43:00.123758 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:00.123711    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:00.162690 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:00.162657    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" event={"ID":"c2d0f487-e80f-42d7-bfdf-ce63c922bec4","Type":"ContainerStarted","Data":"780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5"}
Sep 19 09:43:00.162859 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:00.162713    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" event={"ID":"c2d0f487-e80f-42d7-bfdf-ce63c922bec4","Type":"ContainerStarted","Data":"09791c5600b377b9c16cdb0e46a6788c46f95e31e1a732cb5b29837b8df86b77"}
Sep 19 09:43:00.191606 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:00.191548    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" podStartSLOduration=2.191530171 podStartE2EDuration="2.191530171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:43:00.191081961 +0000 UTC m=+2193.877887261" watchObservedRunningTime="2026-09-19 09:43:00.191530171 +0000 UTC m=+2193.878335525"
Sep 19 09:43:01.003901 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:01.003849    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:01.054000 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:01.053971    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:01.058675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:01.058650    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:01.167826 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:01.167794    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:01.168896 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:01.168875    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:01.970209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:01.970174    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:02.297765 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:02.297647    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:02.445985 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:02.445951    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:03.371074 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:03.371037    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:04.511228 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:04.511190    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:05.396491 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:05.396461    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:06.225587 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:06.225556    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:06.293419 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:06.293382    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:06.872729 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:06.872690    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:06.990069 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:06.990034    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:07.060625 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.060588    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"]
Sep 19 09:43:07.063922 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.063906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.066877 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.066825    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e10409\""
Sep 19 09:43:07.067029 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.066918    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e10409-dockercfg-ck9c7\""
Sep 19 09:43:07.079404 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.079378    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"]
Sep 19 09:43:07.154294 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.154259    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"]
Sep 19 09:43:07.157732 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.157714    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.168370 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.168344    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"]
Sep 19 09:43:07.182525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.182497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhdcm\" (UniqueName: \"kubernetes.io/projected/2046ef60-4f0c-48ed-b65b-32c840fafe27-kube-api-access-fhdcm\") pod \"payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.182709 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.182591    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2046ef60-4f0c-48ed-b65b-32c840fafe27-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.284025 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.283987    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59ef659-1568-436e-bc9c-1ff5e223f08e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.284551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.284039    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhdcm\" (UniqueName: \"kubernetes.io/projected/2046ef60-4f0c-48ed-b65b-32c840fafe27-kube-api-access-fhdcm\") pod \"payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.284551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.284174    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp2nx\" (UniqueName: \"kubernetes.io/projected/c59ef659-1568-436e-bc9c-1ff5e223f08e-kube-api-access-jp2nx\") pod \"payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.284551 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.284237    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2046ef60-4f0c-48ed-b65b-32c840fafe27-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.285035 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.285011    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2046ef60-4f0c-48ed-b65b-32c840fafe27-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.292913 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.292894    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhdcm\" (UniqueName: \"kubernetes.io/projected/2046ef60-4f0c-48ed-b65b-32c840fafe27-kube-api-access-fhdcm\") pod \"payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.374124 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.374058    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:07.384920 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.384887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59ef659-1568-436e-bc9c-1ff5e223f08e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.385046 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.384986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp2nx\" (UniqueName: \"kubernetes.io/projected/c59ef659-1568-436e-bc9c-1ff5e223f08e-kube-api-access-jp2nx\") pod \"payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.385822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.385797    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59ef659-1568-436e-bc9c-1ff5e223f08e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.402680 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.402651    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp2nx\" (UniqueName: \"kubernetes.io/projected/c59ef659-1568-436e-bc9c-1ff5e223f08e-kube-api-access-jp2nx\") pod \"payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.471273 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.471239    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:07.661506 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:43:07.661469    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2046ef60_4f0c_48ed_b65b_32c840fafe27.slice/crio-b552990107ba061c96d49767c6dfa0e4d72b1b735cf0956d795700a8f377013d WatchSource:0}: Error finding container b552990107ba061c96d49767c6dfa0e4d72b1b735cf0956d795700a8f377013d: Status 404 returned error can't find the container with id b552990107ba061c96d49767c6dfa0e4d72b1b735cf0956d795700a8f377013d
Sep 19 09:43:07.662518 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.662485    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"]
Sep 19 09:43:07.750127 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:07.750072    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"]
Sep 19 09:43:07.752143 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:43:07.752120    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59ef659_1568_436e_bc9c_1ff5e223f08e.slice/crio-86e7b462bdef8cfbf4a45a7ab1b21575b2b3eab69ec9cbcc961e0bfaf952b47d WatchSource:0}: Error finding container 86e7b462bdef8cfbf4a45a7ab1b21575b2b3eab69ec9cbcc961e0bfaf952b47d: Status 404 returned error can't find the container with id 86e7b462bdef8cfbf4a45a7ab1b21575b2b3eab69ec9cbcc961e0bfaf952b47d
Sep 19 09:43:08.204973 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.204934    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j" event={"ID":"2046ef60-4f0c-48ed-b65b-32c840fafe27","Type":"ContainerStarted","Data":"ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1"}
Sep 19 09:43:08.205183 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.204999    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j" event={"ID":"2046ef60-4f0c-48ed-b65b-32c840fafe27","Type":"ContainerStarted","Data":"b552990107ba061c96d49767c6dfa0e4d72b1b735cf0956d795700a8f377013d"}
Sep 19 09:43:08.205183 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.205030    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:08.206594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.206559    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp" event={"ID":"c59ef659-1568-436e-bc9c-1ff5e223f08e","Type":"ContainerStarted","Data":"426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af"}
Sep 19 09:43:08.206728 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.206599    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp" event={"ID":"c59ef659-1568-436e-bc9c-1ff5e223f08e","Type":"ContainerStarted","Data":"86e7b462bdef8cfbf4a45a7ab1b21575b2b3eab69ec9cbcc961e0bfaf952b47d"}
Sep 19 09:43:08.206728 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.206688    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:08.223400 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.223332    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j" podStartSLOduration=1.223311252 podStartE2EDuration="1.223311252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:43:08.2227199 +0000 UTC m=+2201.909525195" watchObservedRunningTime="2026-09-19 09:43:08.223311252 +0000 UTC m=+2201.910116553"
Sep 19 09:43:08.242077 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:08.242024    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp" podStartSLOduration=1.242008376 podStartE2EDuration="1.242008376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:43:08.240152911 +0000 UTC m=+2201.926958208" watchObservedRunningTime="2026-09-19 09:43:08.242008376 +0000 UTC m=+2201.928813721"
Sep 19 09:43:10.036681 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:10.036647    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:43:19.214353 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:19.214312    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:43:19.214729 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:19.214392    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:43:21.275572 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:21.275538    2581 generic.go:356] "Generic (PLEG): container finished" podID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerID="780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5" exitCode=0
Sep 19 09:43:21.275997 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:21.275597    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" event={"ID":"c2d0f487-e80f-42d7-bfdf-ce63c922bec4","Type":"ContainerDied","Data":"780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5"}
Sep 19 09:43:21.275997 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:21.275992    2581 scope.go:117] "RemoveContainer" containerID="780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5"
Sep 19 09:43:21.277171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:21.277142    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:43:21.277269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:21.277219    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:43:21.277269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:21.277256    2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 19 09:43:22.282510 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:22.282477    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" event={"ID":"c2d0f487-e80f-42d7-bfdf-ce63c922bec4","Type":"ContainerStarted","Data":"c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec"}
Sep 19 09:43:23.053597 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:23.053568    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:23.053773 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:23.053733    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 19 09:43:23.053819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:23.053780    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 19 09:43:24.053887 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:24.053846    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 19 09:43:24.054313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:24.053911    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 19 09:43:25.053769 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:25.053734    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 19 09:43:25.053977 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:25.053795    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 19 09:43:26.054405 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:26.054365    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 19 09:43:26.054810 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:26.054429    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 19 09:43:27.053903 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:27.053862    2581 patch_prober.go:28] interesting pod/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 19 09:43:27.054128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:27.053924    2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 19 09:43:28.059009 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:28.058976    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:28.059438 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:28.059380    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:28.060323 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:28.060301    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:43:28.816477 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:28.816422    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:29.681140 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:29.681080    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:30.416041 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:30.415999    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:31.202471 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:31.202432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:31.876076 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:31.876030    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:32.310532 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:32.310443    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:37.636492 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:37.636462    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp_c59ef659-1568-436e-bc9c-1ff5e223f08e/payload-processing/0.log"
Sep 19 09:43:44.877868 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.877838    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"]
Sep 19 09:43:44.882413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.882393    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:44.886514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.886492    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6fd1a9\"/\"default-dockercfg-hlrxr\""
Sep 19 09:43:44.886650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.886491    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6fd1a9\"/\"kube-root-ca.crt\""
Sep 19 09:43:44.886650 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.886497    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-6fd1a9\"/\"ipp-route-6fd1a9-kserve-self-signed-certs\""
Sep 19 09:43:44.886791 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.886534    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-6fd1a9\"/\"openshift-service-ca.crt\""
Sep 19 09:43:44.891236 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.891214    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"]
Sep 19 09:43:44.903413 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:44.903383    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:43:45.030145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.030082    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thfc9\" (UniqueName: \"kubernetes.io/projected/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-kube-api-access-thfc9\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.030145 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.030148    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-home\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.030434 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.030180    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-dshm\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.030434 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.030233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-model-cache\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.030434 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.030301    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tmp-dir\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.030434 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.030357    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tls-certs\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131024 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.130932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tmp-dir\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131024 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.130975    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tls-certs\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.131041    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thfc9\" (UniqueName: \"kubernetes.io/projected/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-kube-api-access-thfc9\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.131066    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-home\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.131127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-dshm\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.131206    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-model-cache\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.131445    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tmp-dir\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.131501    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-home\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.131597 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.131511    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-model-cache\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.133643 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.133613    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-dshm\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.133884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.133863    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tls-certs\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.140738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.140713    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thfc9\" (UniqueName: \"kubernetes.io/projected/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-kube-api-access-thfc9\") pod \"ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") " pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.191896 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.191832    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:45.471049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:45.471020    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"]
Sep 19 09:43:45.473239 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:43:45.473208    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd2cfeb0_92e7_4373_be6b_43520ddb0c16.slice/crio-62dbc6e5e33b7ca2c827da52ae998b658640d37ed2242f83a881e1736ee82fb2 WatchSource:0}: Error finding container 62dbc6e5e33b7ca2c827da52ae998b658640d37ed2242f83a881e1736ee82fb2: Status 404 returned error can't find the container with id 62dbc6e5e33b7ca2c827da52ae998b658640d37ed2242f83a881e1736ee82fb2
Sep 19 09:43:46.410455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:46.410413    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn" event={"ID":"cd2cfeb0-92e7-4373-be6b-43520ddb0c16","Type":"ContainerStarted","Data":"d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878"}
Sep 19 09:43:46.410857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:46.410462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn" event={"ID":"cd2cfeb0-92e7-4373-be6b-43520ddb0c16","Type":"ContainerStarted","Data":"62dbc6e5e33b7ca2c827da52ae998b658640d37ed2242f83a881e1736ee82fb2"}
Sep 19 09:43:46.429213 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:46.429161    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn" podStartSLOduration=2.429143852 podStartE2EDuration="2.429143852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:43:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:43:46.428494283 +0000 UTC m=+2240.115299581" watchObservedRunningTime="2026-09-19 09:43:46.429143852 +0000 UTC m=+2240.115949150"
Sep 19 09:43:48.875982 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:48.875894    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:43:55.192431 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:55.192392    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:55.192431 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:55.192444    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:55.205187 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:55.205155    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:43:55.465351 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:43:55.465259    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:44:04.670936 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:04.670900    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:44:10.934340 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:10.934314    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp_c59ef659-1568-436e-bc9c-1ff5e223f08e/payload-processing/0.log"
Sep 19 09:44:11.055891 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:11.055867    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp_c59ef659-1568-436e-bc9c-1ff5e223f08e/payload-processing/0.log"
Sep 19 09:44:11.332622 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:11.332532    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:12.407795 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:12.407762    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:13.851809 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:13.851773    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:14.615239 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:14.615202    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:14.721257 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:14.721227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:14.918628 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:14.918595    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:15.694727 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:15.694671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:16.514410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:16.514375    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:17.332704 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:17.332666    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:18.011539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.011496    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"]
Sep 19 09:44:18.015649 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.015628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.018184 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.018149    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-9f4a5b-serving-cert\""
Sep 19 09:44:18.027508 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.027482    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"]
Sep 19 09:44:18.053137 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.053057    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:18.115920 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.115826    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:18.121903 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.121876    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.122051 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.121907    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpntc\" (UniqueName: \"kubernetes.io/projected/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-kube-api-access-xpntc\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.122051 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.121933    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.122051 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.121995    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.223530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.223487    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.223530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.223531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpntc\" (UniqueName: \"kubernetes.io/projected/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-kube-api-access-xpntc\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.223815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.223556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.223815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.223585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.223815 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:44:18.223716    2581 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-serving-cert: secret "maas-api-e2e-ipp-cleanup-9f4a5b-serving-cert" not found
Sep 19 09:44:18.223815 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:44:18.223799    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls podName:efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f nodeName:}" failed. No retries permitted until 2026-09-19 09:44:18.723782314 +0000 UTC m=+2272.410587612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z" (UID: "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f") : secret "maas-api-e2e-ipp-cleanup-9f4a5b-serving-cert" not found
Sep 19 09:44:18.224368 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.224345    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.226309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.226286    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.232560 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.232529    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpntc\" (UniqueName: \"kubernetes.io/projected/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-kube-api-access-xpntc\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.321831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.321794    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:18.574915 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.574876    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:18.676149 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.676094    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:18.729566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.729534    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.732172 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.732154    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:18.928376 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:18.928341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:19.204340 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:19.204315    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"]
Sep 19 09:44:19.206630 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:44:19.206602    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefea09bf_e3b4_4f1d_88ce_3fd8d0586a8f.slice/crio-58dab1f6d3cb79b41f8d3e28181017b82fcaf3248a773f64a3432443652d4ceb WatchSource:0}: Error finding container 58dab1f6d3cb79b41f8d3e28181017b82fcaf3248a773f64a3432443652d4ceb: Status 404 returned error can't find the container with id 58dab1f6d3cb79b41f8d3e28181017b82fcaf3248a773f64a3432443652d4ceb
Sep 19 09:44:19.207946 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:19.207927    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 09:44:19.577704 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:19.577672    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z" event={"ID":"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f","Type":"ContainerStarted","Data":"58dab1f6d3cb79b41f8d3e28181017b82fcaf3248a773f64a3432443652d4ceb"}
Sep 19 09:44:20.583966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:20.583926    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z" event={"ID":"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f","Type":"ContainerStarted","Data":"ea42969eedf71fb33463de7edcfe4c80ef27a0435c1a9a3e60dc279705b5da92"}
Sep 19 09:44:20.584411 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:20.584059    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:20.604149 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:20.604056    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z" podStartSLOduration=3.244959394 podStartE2EDuration="3.604034825s" totalImagesPullingTime="359.075431ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:44:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:44:20.601457641 +0000 UTC m=+2274.288262949" watchObservedRunningTime="2026-09-19 09:44:20.604034825 +0000 UTC m=+2274.290840126"
Sep 19 09:44:20.984776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:20.984742    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:24.373452 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:24.373420    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:44:26.594974 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:26.594948    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:39.097593 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:39.097555    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:40.547547 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:40.547512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:41.289847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:41.289815    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:42.117608 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:42.117579    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:42.893540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:42.893491    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:43.698634 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:43.698603    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:44.487822 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:44.487782    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:44.614174 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:44.614141    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:46.004598 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:46.004568    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:46.721123 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:46.721067    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:47.464548 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:47.464513    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:48.220408 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:48.220372    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:48.913312 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:48.913269    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:49.615373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:49.615318    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:54.998727 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:54.998694    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:56.056283 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.056244    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:56.598619 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.598587    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"]
Sep 19 09:44:56.598815 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.598788    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z" podUID="efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" containerName="maas-api" containerID="cri-o://ea42969eedf71fb33463de7edcfe4c80ef27a0435c1a9a3e60dc279705b5da92" gracePeriod=30
Sep 19 09:44:56.628661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.628633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:56.696579 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.696547    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:56.760387 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.760344    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:56.770193 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.770165    2581 generic.go:356] "Generic (PLEG): container finished" podID="efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" containerID="ea42969eedf71fb33463de7edcfe4c80ef27a0435c1a9a3e60dc279705b5da92" exitCode=0
Sep 19 09:44:56.770332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.770245    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z" event={"ID":"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f","Type":"ContainerDied","Data":"ea42969eedf71fb33463de7edcfe4c80ef27a0435c1a9a3e60dc279705b5da92"}
Sep 19 09:44:56.831794 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.831765    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:56.845480 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.845456    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:56.892808 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.892767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:56.987667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.987640    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls\") pod \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") "
Sep 19 09:44:56.987848 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.987674    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-metrics-certs\") pod \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") "
Sep 19 09:44:56.987848 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.987700    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-openshift-service-ca\") pod \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") "
Sep 19 09:44:56.987848 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.987742    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpntc\" (UniqueName: \"kubernetes.io/projected/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-kube-api-access-xpntc\") pod \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\" (UID: \"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f\") "
Sep 19 09:44:56.988281 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.988241    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" (UID: "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:44:56.990332 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.990295    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-kube-api-access-xpntc" (OuterVolumeSpecName: "kube-api-access-xpntc") pod "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" (UID: "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f"). InnerVolumeSpecName "kube-api-access-xpntc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:44:56.990459 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.990300    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" (UID: "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:44:56.990655 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:56.990632    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" (UID: "efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:44:57.089390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.089343    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:44:57.089390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.089386    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:44:57.089390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.089396    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:44:57.089390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.089404    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpntc\" (UniqueName: \"kubernetes.io/projected/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f-kube-api-access-xpntc\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:44:57.214555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.214465    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:57.314722 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.314685    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:57.776739 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.776710    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"
Sep 19 09:44:57.776914 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.776754    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z" event={"ID":"efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f","Type":"ContainerDied","Data":"58dab1f6d3cb79b41f8d3e28181017b82fcaf3248a773f64a3432443652d4ceb"}
Sep 19 09:44:57.776914 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.776820    2581 scope.go:117] "RemoveContainer" containerID="ea42969eedf71fb33463de7edcfe4c80ef27a0435c1a9a3e60dc279705b5da92"
Sep 19 09:44:57.804059 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.804005    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"]
Sep 19 09:44:57.805219 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:57.805197    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-9f4a5b-77b4d56b9c-xs47z"]
Sep 19 09:44:58.011298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:58.011258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:58.797863 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:58.797828    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:44:58.831176 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:58.831132    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" path="/var/lib/kubelet/pods/efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f/volumes"
Sep 19 09:44:59.631546 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:44:59.631498    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:00.145479 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.145446    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"]
Sep 19 09:45:00.145884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.145832    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" containerName="maas-api"
Sep 19 09:45:00.145884 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.145843    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" containerName="maas-api"
Sep 19 09:45:00.145957 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.145920    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="efea09bf-e3b4-4f1d-88ce-3fd8d0586a8f" containerName="maas-api"
Sep 19 09:45:00.150468 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.150441    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"]
Sep 19 09:45:00.150645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.150560    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"
Sep 19 09:45:00.152921 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.152899    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v7qlc\""
Sep 19 09:45:00.153524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.153508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"
Sep 19 09:45:00.162725 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.162697    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"]
Sep 19 09:45:00.164330 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.164307    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"
Sep 19 09:45:00.165093 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.165074    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"]
Sep 19 09:45:00.173361 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.173339    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"
Sep 19 09:45:00.456830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.456741    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"]
Sep 19 09:45:00.459056 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:45:00.459029    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7012018a_c238_43e4_b36e_4a584671d890.slice/crio-0c053749f5e35c6110dc53e41f45de8b53ea397322c5b4d3f16041db9c9e3944 WatchSource:0}: Error finding container 0c053749f5e35c6110dc53e41f45de8b53ea397322c5b4d3f16041db9c9e3944: Status 404 returned error can't find the container with id 0c053749f5e35c6110dc53e41f45de8b53ea397322c5b4d3f16041db9c9e3944
Sep 19 09:45:00.459230 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.459210    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"]
Sep 19 09:45:00.460966 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:45:00.460891    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3136399c_1329_425d_86d4_31b5adc7a947.slice/crio-431447a27f353b3b9ffc4176bd8e7f64c5fd5bf9b4c3d3368310b5cd15da39d5 WatchSource:0}: Error finding container 431447a27f353b3b9ffc4176bd8e7f64c5fd5bf9b4c3d3368310b5cd15da39d5: Status 404 returned error can't find the container with id 431447a27f353b3b9ffc4176bd8e7f64c5fd5bf9b4c3d3368310b5cd15da39d5
Sep 19 09:45:00.550548 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.550505    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:00.794955 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.794923    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss" event={"ID":"7012018a-c238-43e4-b36e-4a584671d890","Type":"ContainerStarted","Data":"0c053749f5e35c6110dc53e41f45de8b53ea397322c5b4d3f16041db9c9e3944"}
Sep 19 09:45:00.797061 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:00.797022    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k" event={"ID":"3136399c-1329-425d-86d4-31b5adc7a947","Type":"ContainerStarted","Data":"431447a27f353b3b9ffc4176bd8e7f64c5fd5bf9b4c3d3368310b5cd15da39d5"}
Sep 19 09:45:01.224321 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:01.224284    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:45:01.803552 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:01.803454    2581 generic.go:356] "Generic (PLEG): container finished" podID="7012018a-c238-43e4-b36e-4a584671d890" containerID="c198ecca37101faf5edd3167d09f308e98da598a575f49493ff6a7bc9399e269" exitCode=0
Sep 19 09:45:01.803730 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:01.803548    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss" event={"ID":"7012018a-c238-43e4-b36e-4a584671d890","Type":"ContainerDied","Data":"c198ecca37101faf5edd3167d09f308e98da598a575f49493ff6a7bc9399e269"}
Sep 19 09:45:01.805165 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:01.805144    2581 generic.go:356] "Generic (PLEG): container finished" podID="3136399c-1329-425d-86d4-31b5adc7a947" containerID="ce440418c4e90f1a6d65da18a9dc75132db7f06de9df94fa7644e1ddce5c249e" exitCode=0
Sep 19 09:45:01.805288 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:01.805172    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k" event={"ID":"3136399c-1329-425d-86d4-31b5adc7a947","Type":"ContainerDied","Data":"ce440418c4e90f1a6d65da18a9dc75132db7f06de9df94fa7644e1ddce5c249e"}
Sep 19 09:45:02.968835 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:02.968808    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"
Sep 19 09:45:02.972031 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:02.972007    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"
Sep 19 09:45:03.381665 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.381633    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:03.482340 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.482305    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:03.659722 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.659627    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:03.817549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.817523    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"
Sep 19 09:45:03.817549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.817542    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss" event={"ID":"7012018a-c238-43e4-b36e-4a584671d890","Type":"ContainerDied","Data":"0c053749f5e35c6110dc53e41f45de8b53ea397322c5b4d3f16041db9c9e3944"}
Sep 19 09:45:03.817824 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.817627    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c053749f5e35c6110dc53e41f45de8b53ea397322c5b4d3f16041db9c9e3944"
Sep 19 09:45:03.819202 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.819176    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k" event={"ID":"3136399c-1329-425d-86d4-31b5adc7a947","Type":"ContainerDied","Data":"431447a27f353b3b9ffc4176bd8e7f64c5fd5bf9b4c3d3368310b5cd15da39d5"}
Sep 19 09:45:03.819346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.819208    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"
Sep 19 09:45:03.819346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:03.819223    2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="431447a27f353b3b9ffc4176bd8e7f64c5fd5bf9b4c3d3368310b5cd15da39d5"
Sep 19 09:45:07.377115 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:07.377073    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:08.225575 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:08.225538    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:08.926663 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:08.926630    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:09.776344 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:09.776310    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:10.910696 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:10.910668    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:11.958745 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:11.958704    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:20.584646 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:20.584613    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:45:22.919423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:22.919391    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"]
Sep 19 09:45:22.919852 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:22.919598    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" podUID="cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" containerName="maas-api" containerID="cri-o://fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04" gracePeriod=30
Sep 19 09:45:22.959030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:22.958996    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:22.965425 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:22.965401    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"]
Sep 19 09:45:22.969342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:22.969318    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-6fd1a9-29830185-2tf2k"]
Sep 19 09:45:23.009787 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.009753    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"]
Sep 19 09:45:23.010083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.010022    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb" podUID="ff78d409-13d1-4c4c-b939-962ae9d46d63" containerName="payload-pre-processing" containerID="cri-o://360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d" gracePeriod=30
Sep 19 09:45:23.046995 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.046953    2581 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" podUID="cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.84:8443/health\": dial tcp 10.132.0.84:8443: connect: connection refused"
Sep 19 09:45:23.058618 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.058570    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:23.162056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.162000    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:23.164592 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.164569    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:45:23.226320 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.226281    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-metrics-certs\") pod \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") "
Sep 19 09:45:23.226525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.226335    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-openshift-service-ca\") pod \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") "
Sep 19 09:45:23.226525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.226371    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls\") pod \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") "
Sep 19 09:45:23.226525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.226437    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dp9x\" (UniqueName: \"kubernetes.io/projected/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-kube-api-access-5dp9x\") pod \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\" (UID: \"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7\") "
Sep 19 09:45:23.226814 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.226783    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" (UID: "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:45:23.228734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.228710    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" (UID: "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:45:23.228817 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.228749    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" (UID: "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:45:23.228895 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.228879    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-kube-api-access-5dp9x" (OuterVolumeSpecName: "kube-api-access-5dp9x") pod "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" (UID: "cfbaced6-9743-4b4d-b5bb-1c810ddd91a7"). InnerVolumeSpecName "kube-api-access-5dp9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:45:23.239362 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.239331    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:23.274987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.274961    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:45:23.327980 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.327951    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dp9x\" (UniqueName: \"kubernetes.io/projected/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-kube-api-access-5dp9x\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:23.327980 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.327977    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:23.328215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.327990    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:23.328215 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.328000    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:23.428844 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.428809    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff78d409-13d1-4c4c-b939-962ae9d46d63-plugins-config-volume\") pod \"ff78d409-13d1-4c4c-b939-962ae9d46d63\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") "
Sep 19 09:45:23.429026 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.428878    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghm2r\" (UniqueName: \"kubernetes.io/projected/ff78d409-13d1-4c4c-b939-962ae9d46d63-kube-api-access-ghm2r\") pod \"ff78d409-13d1-4c4c-b939-962ae9d46d63\" (UID: \"ff78d409-13d1-4c4c-b939-962ae9d46d63\") "
Sep 19 09:45:23.429269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.429242    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff78d409-13d1-4c4c-b939-962ae9d46d63-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff78d409-13d1-4c4c-b939-962ae9d46d63" (UID: "ff78d409-13d1-4c4c-b939-962ae9d46d63"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:45:23.431184 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.431159    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff78d409-13d1-4c4c-b939-962ae9d46d63-kube-api-access-ghm2r" (OuterVolumeSpecName: "kube-api-access-ghm2r") pod "ff78d409-13d1-4c4c-b939-962ae9d46d63" (UID: "ff78d409-13d1-4c4c-b939-962ae9d46d63"). InnerVolumeSpecName "kube-api-access-ghm2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:45:23.530457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.530368    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff78d409-13d1-4c4c-b939-962ae9d46d63-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:23.530457 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.530405    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghm2r\" (UniqueName: \"kubernetes.io/projected/ff78d409-13d1-4c4c-b939-962ae9d46d63-kube-api-access-ghm2r\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:23.580591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.580558    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:23.678159 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.678118    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:23.934675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.934542    2581 generic.go:356] "Generic (PLEG): container finished" podID="cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" containerID="fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04" exitCode=0
Sep 19 09:45:23.934675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.934581    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"
Sep 19 09:45:23.934675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.934623    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" event={"ID":"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7","Type":"ContainerDied","Data":"fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04"}
Sep 19 09:45:23.934675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.934669    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg" event={"ID":"cfbaced6-9743-4b4d-b5bb-1c810ddd91a7","Type":"ContainerDied","Data":"1679ce6655ade566770591fb3ae0a5c7253d2ef991be3d22f544d889ec8c071f"}
Sep 19 09:45:23.935184 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.934691    2581 scope.go:117] "RemoveContainer" containerID="fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04"
Sep 19 09:45:23.936789 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.936736    2581 generic.go:356] "Generic (PLEG): container finished" podID="ff78d409-13d1-4c4c-b939-962ae9d46d63" containerID="360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d" exitCode=0
Sep 19 09:45:23.937238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.936795    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"
Sep 19 09:45:23.937238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.936798    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb" event={"ID":"ff78d409-13d1-4c4c-b939-962ae9d46d63","Type":"ContainerDied","Data":"360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d"}
Sep 19 09:45:23.937238 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.937080    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb" event={"ID":"ff78d409-13d1-4c4c-b939-962ae9d46d63","Type":"ContainerDied","Data":"7620969dbe06f227c4abf94989381abff0506fc71d6ef2d31c368f5462935ae6"}
Sep 19 09:45:23.943959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.943941    2581 scope.go:117] "RemoveContainer" containerID="fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04"
Sep 19 09:45:23.944326 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:45:23.944302    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04\": container with ID starting with fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04 not found: ID does not exist" containerID="fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04"
Sep 19 09:45:23.944420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.944334    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04"} err="failed to get container status \"fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04\": rpc error: code = NotFound desc = could not find container \"fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04\": container with ID starting with fba694773f87b4cbd258d2772d9600349e1aeddc3259a9454f0fc5aff062bb04 not found: ID does not exist"
Sep 19 09:45:23.944420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.944353    2581 scope.go:117] "RemoveContainer" containerID="360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d"
Sep 19 09:45:23.953179 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.953161    2581 scope.go:117] "RemoveContainer" containerID="360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d"
Sep 19 09:45:23.953449 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:45:23.953430    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d\": container with ID starting with 360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d not found: ID does not exist" containerID="360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d"
Sep 19 09:45:23.953507 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.953456    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d"} err="failed to get container status \"360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d\": rpc error: code = NotFound desc = could not find container \"360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d\": container with ID starting with 360d7c15ce8fdf777cc1584df2faa242002f0de677c5584e872ee5a01a3b770d not found: ID does not exist"
Sep 19 09:45:23.972856 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.972823    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"]
Sep 19 09:45:23.975069 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:23.975044    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fd1a9-68789877b8-ft5qg"]
Sep 19 09:45:24.011927 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:24.011896    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"]
Sep 19 09:45:24.019573 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:24.019537    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fd1a9-5c758dddbb-sm2vb"]
Sep 19 09:45:24.512306 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:24.512276    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:24.831691 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:24.831609    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3136399c-1329-425d-86d4-31b5adc7a947" path="/var/lib/kubelet/pods/3136399c-1329-425d-86d4-31b5adc7a947/volumes"
Sep 19 09:45:24.831983 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:24.831970    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" path="/var/lib/kubelet/pods/cfbaced6-9743-4b4d-b5bb-1c810ddd91a7/volumes"
Sep 19 09:45:24.832337 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:24.832323    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff78d409-13d1-4c4c-b939-962ae9d46d63" path="/var/lib/kubelet/pods/ff78d409-13d1-4c4c-b939-962ae9d46d63/volumes"
Sep 19 09:45:25.214848 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:25.214813    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:25.941453 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:25.941417    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:28.548361 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:28.548317    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"]
Sep 19 09:45:28.599995 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:28.599956    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:45:33.576932 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:33.576878    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:45:43.568262 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.568193    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn" podUID="cd2cfeb0-92e7-4373-be6b-43520ddb0c16" containerName="main" containerID="cri-o://d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878" gracePeriod=45
Sep 19 09:45:43.824484 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.824427    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:45:43.922015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.921987    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-home\") pod \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") "
Sep 19 09:45:43.922015 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922019    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-model-cache\") pod \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") "
Sep 19 09:45:43.922298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922043    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-dshm\") pod \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") "
Sep 19 09:45:43.922298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922064    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thfc9\" (UniqueName: \"kubernetes.io/projected/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-kube-api-access-thfc9\") pod \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") "
Sep 19 09:45:43.922298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922145    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tmp-dir\") pod \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") "
Sep 19 09:45:43.922298 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922191    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tls-certs\") pod \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\" (UID: \"cd2cfeb0-92e7-4373-be6b-43520ddb0c16\") "
Sep 19 09:45:43.922513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922375    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-home" (OuterVolumeSpecName: "home") pod "cd2cfeb0-92e7-4373-be6b-43520ddb0c16" (UID: "cd2cfeb0-92e7-4373-be6b-43520ddb0c16"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:45:43.922513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922427    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd2cfeb0-92e7-4373-be6b-43520ddb0c16" (UID: "cd2cfeb0-92e7-4373-be6b-43520ddb0c16"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:45:43.922513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.922435    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd2cfeb0-92e7-4373-be6b-43520ddb0c16" (UID: "cd2cfeb0-92e7-4373-be6b-43520ddb0c16"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:45:43.924510 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.924478    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-dshm" (OuterVolumeSpecName: "dshm") pod "cd2cfeb0-92e7-4373-be6b-43520ddb0c16" (UID: "cd2cfeb0-92e7-4373-be6b-43520ddb0c16"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:45:43.924629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.924524    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd2cfeb0-92e7-4373-be6b-43520ddb0c16" (UID: "cd2cfeb0-92e7-4373-be6b-43520ddb0c16"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:45:43.924629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:43.924529    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-kube-api-access-thfc9" (OuterVolumeSpecName: "kube-api-access-thfc9") pod "cd2cfeb0-92e7-4373-be6b-43520ddb0c16" (UID: "cd2cfeb0-92e7-4373-be6b-43520ddb0c16"). InnerVolumeSpecName "kube-api-access-thfc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:45:44.023813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.023772    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:44.023813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.023804    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:44.023813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.023814    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:44.024064 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.023825    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:44.024064 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.023835    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:44.024064 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.023846    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thfc9\" (UniqueName: \"kubernetes.io/projected/cd2cfeb0-92e7-4373-be6b-43520ddb0c16-kube-api-access-thfc9\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:45:44.048594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.048557    2581 generic.go:356] "Generic (PLEG): container finished" podID="cd2cfeb0-92e7-4373-be6b-43520ddb0c16" containerID="d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878" exitCode=0
Sep 19 09:45:44.048757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.048600    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"
Sep 19 09:45:44.048757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.048630    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn" event={"ID":"cd2cfeb0-92e7-4373-be6b-43520ddb0c16","Type":"ContainerDied","Data":"d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878"}
Sep 19 09:45:44.048757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.048663    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn" event={"ID":"cd2cfeb0-92e7-4373-be6b-43520ddb0c16","Type":"ContainerDied","Data":"62dbc6e5e33b7ca2c827da52ae998b658640d37ed2242f83a881e1736ee82fb2"}
Sep 19 09:45:44.048757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.048679    2581 scope.go:117] "RemoveContainer" containerID="d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878"
Sep 19 09:45:44.058134 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.058081    2581 scope.go:117] "RemoveContainer" containerID="d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878"
Sep 19 09:45:44.058456 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:45:44.058434    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878\": container with ID starting with d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878 not found: ID does not exist" containerID="d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878"
Sep 19 09:45:44.058526 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.058466    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878"} err="failed to get container status \"d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878\": rpc error: code = NotFound desc = could not find container \"d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878\": container with ID starting with d79fa1f47171c3f64ffeef86513dc59c07caa4d272cc012dffe05c179fedb878 not found: ID does not exist"
Sep 19 09:45:44.070138 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.070107    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"]
Sep 19 09:45:44.075782 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.075721    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-6fd1a9/ipp-route-6fd1a9-kserve-7bc8c7cf6c-zljnn"]
Sep 19 09:45:44.831340 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:45:44.831312    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd2cfeb0-92e7-4373-be6b-43520ddb0c16" path="/var/lib/kubelet/pods/cd2cfeb0-92e7-4373-be6b-43520ddb0c16/volumes"
Sep 19 09:46:05.262620 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:05.262582    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:05.693020 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:05.692987    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:11.445708 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:11.445678    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:26.507482 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:26.507440    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:26.607430 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:26.607397    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:26.810780 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:26.810681    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:28.093687 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:28.093651    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:28.995623 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:28.995584    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:30.059542 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:30.059512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:30.935182 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:30.935150    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:37.624916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.624870    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:37.641707 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.641676    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"]
Sep 19 09:46:37.641942 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.641895    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp" podUID="c59ef659-1568-436e-bc9c-1ff5e223f08e" containerName="payload-processing" containerID="cri-o://426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af" gracePeriod=30
Sep 19 09:46:37.661047 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.660900    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"]
Sep 19 09:46:37.661306 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.661257    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j" podUID="2046ef60-4f0c-48ed-b65b-32c840fafe27" containerName="payload-pre-processing" containerID="cri-o://ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1" gracePeriod=30
Sep 19 09:46:37.726462 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.726432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:37.826216 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.826170    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:37.911395 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.911367    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:46:37.925167 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.925130    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:37.939959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:37.939932    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:46:38.000638 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.000587    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp2nx\" (UniqueName: \"kubernetes.io/projected/c59ef659-1568-436e-bc9c-1ff5e223f08e-kube-api-access-jp2nx\") pod \"c59ef659-1568-436e-bc9c-1ff5e223f08e\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") "
Sep 19 09:46:38.000830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.000715    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhdcm\" (UniqueName: \"kubernetes.io/projected/2046ef60-4f0c-48ed-b65b-32c840fafe27-kube-api-access-fhdcm\") pod \"2046ef60-4f0c-48ed-b65b-32c840fafe27\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") "
Sep 19 09:46:38.000830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.000736    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2046ef60-4f0c-48ed-b65b-32c840fafe27-plugins-config-volume\") pod \"2046ef60-4f0c-48ed-b65b-32c840fafe27\" (UID: \"2046ef60-4f0c-48ed-b65b-32c840fafe27\") "
Sep 19 09:46:38.000830 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.000761    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59ef659-1568-436e-bc9c-1ff5e223f08e-plugins-config-volume\") pod \"c59ef659-1568-436e-bc9c-1ff5e223f08e\" (UID: \"c59ef659-1568-436e-bc9c-1ff5e223f08e\") "
Sep 19 09:46:38.001210 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.001182    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2046ef60-4f0c-48ed-b65b-32c840fafe27-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2046ef60-4f0c-48ed-b65b-32c840fafe27" (UID: "2046ef60-4f0c-48ed-b65b-32c840fafe27"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:46:38.001313 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.001242    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c59ef659-1568-436e-bc9c-1ff5e223f08e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c59ef659-1568-436e-bc9c-1ff5e223f08e" (UID: "c59ef659-1568-436e-bc9c-1ff5e223f08e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:46:38.003074 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.003047    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c59ef659-1568-436e-bc9c-1ff5e223f08e-kube-api-access-jp2nx" (OuterVolumeSpecName: "kube-api-access-jp2nx") pod "c59ef659-1568-436e-bc9c-1ff5e223f08e" (UID: "c59ef659-1568-436e-bc9c-1ff5e223f08e"). InnerVolumeSpecName "kube-api-access-jp2nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:46:38.003074 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.003053    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2046ef60-4f0c-48ed-b65b-32c840fafe27-kube-api-access-fhdcm" (OuterVolumeSpecName: "kube-api-access-fhdcm") pod "2046ef60-4f0c-48ed-b65b-32c840fafe27" (UID: "2046ef60-4f0c-48ed-b65b-32c840fafe27"). InnerVolumeSpecName "kube-api-access-fhdcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:46:38.102032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.101978    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhdcm\" (UniqueName: \"kubernetes.io/projected/2046ef60-4f0c-48ed-b65b-32c840fafe27-kube-api-access-fhdcm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:38.102032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.102026    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2046ef60-4f0c-48ed-b65b-32c840fafe27-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:38.102032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.102036    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c59ef659-1568-436e-bc9c-1ff5e223f08e-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:38.102032 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.102045    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp2nx\" (UniqueName: \"kubernetes.io/projected/c59ef659-1568-436e-bc9c-1ff5e223f08e-kube-api-access-jp2nx\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:38.216513 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.216408    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:38.316645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.316606    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:38.322314 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.322279    2581 generic.go:356] "Generic (PLEG): container finished" podID="2046ef60-4f0c-48ed-b65b-32c840fafe27" containerID="ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1" exitCode=0
Sep 19 09:46:38.322496 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.322337    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"
Sep 19 09:46:38.322496 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.322366    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j" event={"ID":"2046ef60-4f0c-48ed-b65b-32c840fafe27","Type":"ContainerDied","Data":"ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1"}
Sep 19 09:46:38.322496 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.322462    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j" event={"ID":"2046ef60-4f0c-48ed-b65b-32c840fafe27","Type":"ContainerDied","Data":"b552990107ba061c96d49767c6dfa0e4d72b1b735cf0956d795700a8f377013d"}
Sep 19 09:46:38.322496 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.322493    2581 scope.go:117] "RemoveContainer" containerID="ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1"
Sep 19 09:46:38.323780 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.323762    2581 generic.go:356] "Generic (PLEG): container finished" podID="c59ef659-1568-436e-bc9c-1ff5e223f08e" containerID="426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af" exitCode=0
Sep 19 09:46:38.323845 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.323807    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp" event={"ID":"c59ef659-1568-436e-bc9c-1ff5e223f08e","Type":"ContainerDied","Data":"426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af"}
Sep 19 09:46:38.323845 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.323829    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"
Sep 19 09:46:38.323959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.323845    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp" event={"ID":"c59ef659-1568-436e-bc9c-1ff5e223f08e","Type":"ContainerDied","Data":"86e7b462bdef8cfbf4a45a7ab1b21575b2b3eab69ec9cbcc961e0bfaf952b47d"}
Sep 19 09:46:38.332444 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.332424    2581 scope.go:117] "RemoveContainer" containerID="ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1"
Sep 19 09:46:38.332751 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:46:38.332728    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1\": container with ID starting with ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1 not found: ID does not exist" containerID="ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1"
Sep 19 09:46:38.332831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.332761    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1"} err="failed to get container status \"ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1\": rpc error: code = NotFound desc = could not find container \"ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1\": container with ID starting with ba1de7eb24188c15556426d996e2d00e6e406df0939bebbfe567c2ad265416b1 not found: ID does not exist"
Sep 19 09:46:38.332831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.332780    2581 scope.go:117] "RemoveContainer" containerID="426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af"
Sep 19 09:46:38.341309 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.341289    2581 scope.go:117] "RemoveContainer" containerID="426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af"
Sep 19 09:46:38.341598 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:46:38.341577    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af\": container with ID starting with 426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af not found: ID does not exist" containerID="426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af"
Sep 19 09:46:38.341676 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.341606    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af"} err="failed to get container status \"426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af\": rpc error: code = NotFound desc = could not find container \"426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af\": container with ID starting with 426849b1760072ec4315811ebcaf9bf7013c91909f823a9f5da44bb406a2c2af not found: ID does not exist"
Sep 19 09:46:38.347919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.347892    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"]
Sep 19 09:46:38.350207 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.350183    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e10409-6f85544b45-g24gp"]
Sep 19 09:46:38.359861 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.359839    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"]
Sep 19 09:46:38.363256 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.363234    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e10409-5874c687db-tgp7j"]
Sep 19 09:46:38.831536 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.831502    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2046ef60-4f0c-48ed-b65b-32c840fafe27" path="/var/lib/kubelet/pods/2046ef60-4f0c-48ed-b65b-32c840fafe27/volumes"
Sep 19 09:46:38.831911 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.831850    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c59ef659-1568-436e-bc9c-1ff5e223f08e" path="/var/lib/kubelet/pods/c59ef659-1568-436e-bc9c-1ff5e223f08e/volumes"
Sep 19 09:46:38.985525 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:38.985437    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:39.813635 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:39.813598    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:40.981372 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:40.981336    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:46:44.149580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:44.149534    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:44.248504 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:44.248474    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:44.423060 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:44.422976    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"]
Sep 19 09:46:44.423269 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:44.423249    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy" containerID="cri-o://c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec" gracePeriod=30
Sep 19 09:46:44.444775 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:44.444753    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:44.935410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:44.935373    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:45.912530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:45.912488    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:46.014797 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:46.014767    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:46.780693 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:46.780657    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:47.562644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:47.562560    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:49.669738 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.669715    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:46:49.809157 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809038    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-podinfo\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809157 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809092    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-data\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809157 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809147    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rbm7\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-kube-api-access-8rbm7\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809176    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-socket\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809202    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istiod-ca-cert\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809222    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-credential-socket\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809243    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-token\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809272    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-certs\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809469 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809304    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-envoy\") pod \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\" (UID: \"c2d0f487-e80f-42d7-bfdf-ce63c922bec4\") "
Sep 19 09:46:49.809734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809531    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-data" (OuterVolumeSpecName: "istio-data") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:46:49.809734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809533    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:46:49.809734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809637    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:46:49.809734 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809647    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:46:49.809916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.809795    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:46:49.811602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.811571    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-kube-api-access-8rbm7" (OuterVolumeSpecName: "kube-api-access-8rbm7") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "kube-api-access-8rbm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:46:49.811602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.811589    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 09:46:49.811602 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.811600    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:46:49.811847 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.811734    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-token" (OuterVolumeSpecName: "istio-token") pod "c2d0f487-e80f-42d7-bfdf-ce63c922bec4" (UID: "c2d0f487-e80f-42d7-bfdf-ce63c922bec4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:46:49.910295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910252    2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-token\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910289    2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910298    2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-envoy\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910295 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910306    2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-podinfo\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910314    2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istio-data\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910322    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rbm7\" (UniqueName: \"kubernetes.io/projected/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-kube-api-access-8rbm7\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910331    2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-workload-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910340    2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-istiod-ca-cert\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:49.910580 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:49.910347    2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2d0f487-e80f-42d7-bfdf-ce63c922bec4-credential-socket\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:46:50.386767 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.386734    2581 generic.go:356] "Generic (PLEG): container finished" podID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerID="c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec" exitCode=0
Sep 19 09:46:50.386963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.386796    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"
Sep 19 09:46:50.386963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.386810    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" event={"ID":"c2d0f487-e80f-42d7-bfdf-ce63c922bec4","Type":"ContainerDied","Data":"c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec"}
Sep 19 09:46:50.386963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.386884    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb" event={"ID":"c2d0f487-e80f-42d7-bfdf-ce63c922bec4","Type":"ContainerDied","Data":"09791c5600b377b9c16cdb0e46a6788c46f95e31e1a732cb5b29837b8df86b77"}
Sep 19 09:46:50.386963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.386906    2581 scope.go:117] "RemoveContainer" containerID="c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec"
Sep 19 09:46:50.397069 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.397048    2581 scope.go:117] "RemoveContainer" containerID="780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5"
Sep 19 09:46:50.405311 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.405294    2581 scope.go:117] "RemoveContainer" containerID="c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec"
Sep 19 09:46:50.405596 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:46:50.405574    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec\": container with ID starting with c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec not found: ID does not exist" containerID="c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec"
Sep 19 09:46:50.405644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.405606    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec"} err="failed to get container status \"c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec\": rpc error: code = NotFound desc = could not find container \"c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec\": container with ID starting with c1d22e5290b9ca6503daeedc896462585a52659d63d0fe013583712ef797d8ec not found: ID does not exist"
Sep 19 09:46:50.405644 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.405624    2581 scope.go:117] "RemoveContainer" containerID="780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5"
Sep 19 09:46:50.405880 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:46:50.405866    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5\": container with ID starting with 780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5 not found: ID does not exist" containerID="780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5"
Sep 19 09:46:50.405936 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.405881    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5"} err="failed to get container status \"780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5\": rpc error: code = NotFound desc = could not find container \"780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5\": container with ID starting with 780de6909271c8a9c46c31a475edb37a6ecb08082dbb429471ef25d2239935c5 not found: ID does not exist"
Sep 19 09:46:50.409723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.409699    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"]
Sep 19 09:46:50.411629 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.411606    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-e10409-openshift-default-5ccc8f479d-ghbnb"]
Sep 19 09:46:50.831897 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:50.831812    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" path="/var/lib/kubelet/pods/c2d0f487-e80f-42d7-bfdf-ce63c922bec4/volumes"
Sep 19 09:46:55.316596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:55.316561    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:55.415910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:55.415880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:55.518056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:55.518027    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:55.616972 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:55.616874    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:55.912923 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:55.912880    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:56.062373 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:56.062320    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:56.877314 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:56.877274    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:46:58.486216 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:46:58.486172    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:47:02.228092 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:02.228054    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:02.326909 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:02.326868    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:02.497619 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:02.497517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:03.889829 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:03.889789    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:05.026543 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:05.026512    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:13.441893 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.441858    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:13.442397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.441968    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"]
Sep 19 09:47:13.443603 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.443578    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1afa13-29830185-csbss"]
Sep 19 09:47:13.474959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.474927    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"]
Sep 19 09:47:13.475236 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.475208    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8" podUID="a76a4fc5-4a86-4a21-bc6d-1ff073968f99" containerName="payload-pre-processing" containerID="cri-o://dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e" gracePeriod=30
Sep 19 09:47:13.512321 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:47:13.512292    2581 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda76a4fc5_4a86_4a21_bc6d_1ff073968f99.slice/crio-dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e.scope\": RecentStats: unable to find data in memory cache]"
Sep 19 09:47:13.543274 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.543242    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:13.640953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.640921    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:13.730364 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.730336    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:47:13.746549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.746521    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:13.920231 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.920193    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-plugins-config-volume\") pod \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") "
Sep 19 09:47:13.920412 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.920262    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vml6r\" (UniqueName: \"kubernetes.io/projected/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-kube-api-access-vml6r\") pod \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\" (UID: \"a76a4fc5-4a86-4a21-bc6d-1ff073968f99\") "
Sep 19 09:47:13.920638 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.920617    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a76a4fc5-4a86-4a21-bc6d-1ff073968f99" (UID: "a76a4fc5-4a86-4a21-bc6d-1ff073968f99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:47:13.922588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:13.922566    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-kube-api-access-vml6r" (OuterVolumeSpecName: "kube-api-access-vml6r") pod "a76a4fc5-4a86-4a21-bc6d-1ff073968f99" (UID: "a76a4fc5-4a86-4a21-bc6d-1ff073968f99"). InnerVolumeSpecName "kube-api-access-vml6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:47:14.020929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.020842    2581 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-plugins-config-volume\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:47:14.020929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.020871    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vml6r\" (UniqueName: \"kubernetes.io/projected/a76a4fc5-4a86-4a21-bc6d-1ff073968f99-kube-api-access-vml6r\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:47:14.089839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.089799    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:14.189437 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.189390    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:14.505521 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.505489    2581 generic.go:356] "Generic (PLEG): container finished" podID="a76a4fc5-4a86-4a21-bc6d-1ff073968f99" containerID="dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e" exitCode=0
Sep 19 09:47:14.505987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.505525    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8" event={"ID":"a76a4fc5-4a86-4a21-bc6d-1ff073968f99","Type":"ContainerDied","Data":"dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e"}
Sep 19 09:47:14.505987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.505594    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8" event={"ID":"a76a4fc5-4a86-4a21-bc6d-1ff073968f99","Type":"ContainerDied","Data":"33205dc6046e18a457bab820ce64176f966c2c49dc18f4c78695da77c8d8f97a"}
Sep 19 09:47:14.505987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.505613    2581 scope.go:117] "RemoveContainer" containerID="dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e"
Sep 19 09:47:14.505987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.505535    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"
Sep 19 09:47:14.521092 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.521068    2581 scope.go:117] "RemoveContainer" containerID="dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e"
Sep 19 09:47:14.521410 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:47:14.521393    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e\": container with ID starting with dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e not found: ID does not exist" containerID="dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e"
Sep 19 09:47:14.521466 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.521417    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e"} err="failed to get container status \"dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e\": rpc error: code = NotFound desc = could not find container \"dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e\": container with ID starting with dbb2243e1a948c2d55f9ddf9c3011744b31a3eea28d3c698ce620695ba694b3e not found: ID does not exist"
Sep 19 09:47:14.533092 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.533063    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"]
Sep 19 09:47:14.535769 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.535745    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1afa13-7c9557fd54-r74q8"]
Sep 19 09:47:14.830121 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.830027    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7012018a-c238-43e4-b36e-4a584671d890" path="/var/lib/kubelet/pods/7012018a-c238-43e4-b36e-4a584671d890/volumes"
Sep 19 09:47:14.830397 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:14.830376    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a76a4fc5-4a86-4a21-bc6d-1ff073968f99" path="/var/lib/kubelet/pods/a76a4fc5-4a86-4a21-bc6d-1ff073968f99/volumes"
Sep 19 09:47:16.290639 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:16.290604    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:47:17.456259 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:17.456224    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:20.371218 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:20.371165    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:20.471819 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:20.471780    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:20.656594 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:20.656557    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:20.761702 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:20.761671    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:47:25.896643 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:47:25.896592    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:48:00.702524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:48:00.702486    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:48:47.578418 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:48:47.578375    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:48:48.335116 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:48:48.335072    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:48:49.863470 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:48:49.863432    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:48:51.730792 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:48:51.730757    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:49:00.893605 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:49:00.893569    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:49:00.953540 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:49:00.953506    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:49:27.761030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:49:27.760994    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:49:35.204539 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:49:35.204498    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:49:42.275029 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:49:42.274990    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:49:50.580224 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:49:50.580135    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:50:04.680888 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:50:04.680851    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:50:09.587272 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:50:09.587230    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:50:12.407517 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:50:12.407481    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:51:04.024173 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:04.024136    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:51:04.141839 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:04.141803    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:51:09.239216 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:09.239180    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:51:27.315481 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:27.315451    2581 scope.go:117] "RemoveContainer" containerID="c198ecca37101faf5edd3167d09f308e98da598a575f49493ff6a7bc9399e269"
Sep 19 09:51:27.323886 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:27.323857    2581 scope.go:117] "RemoveContainer" containerID="ce440418c4e90f1a6d65da18a9dc75132db7f06de9df94fa7644e1ddce5c249e"
Sep 19 09:51:43.062686 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:43.062650    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:51:51.380527 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:51.380488    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:51:59.576755 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:51:59.576714    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:52:14.274750 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.274709    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"]
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275061    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd2cfeb0-92e7-4373-be6b-43520ddb0c16" containerName="main"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275078    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd2cfeb0-92e7-4373-be6b-43520ddb0c16" containerName="main"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275113    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275122    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275133    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3136399c-1329-425d-86d4-31b5adc7a947" containerName="cleanup"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275141    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3136399c-1329-425d-86d4-31b5adc7a947" containerName="cleanup"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275161    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff78d409-13d1-4c4c-b939-962ae9d46d63" containerName="payload-pre-processing"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275169    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff78d409-13d1-4c4c-b939-962ae9d46d63" containerName="payload-pre-processing"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275188    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2046ef60-4f0c-48ed-b65b-32c840fafe27" containerName="payload-pre-processing"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275197    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2046ef60-4f0c-48ed-b65b-32c840fafe27" containerName="payload-pre-processing"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275209    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c59ef659-1568-436e-bc9c-1ff5e223f08e" containerName="payload-processing"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275218    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c59ef659-1568-436e-bc9c-1ff5e223f08e" containerName="payload-processing"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275229    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" containerName="maas-api"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275238    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" containerName="maas-api"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275247    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7012018a-c238-43e4-b36e-4a584671d890" containerName="cleanup"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275256    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7012018a-c238-43e4-b36e-4a584671d890" containerName="cleanup"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275273    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a76a4fc5-4a86-4a21-bc6d-1ff073968f99" containerName="payload-pre-processing"
Sep 19 09:52:14.275331 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275282    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a76a4fc5-4a86-4a21-bc6d-1ff073968f99" containerName="payload-pre-processing"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275364    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c59ef659-1568-436e-bc9c-1ff5e223f08e" containerName="payload-processing"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275376    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7012018a-c238-43e4-b36e-4a584671d890" containerName="cleanup"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275388    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275396    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a76a4fc5-4a86-4a21-bc6d-1ff073968f99" containerName="payload-pre-processing"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275406    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd2cfeb0-92e7-4373-be6b-43520ddb0c16" containerName="main"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275414    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3136399c-1329-425d-86d4-31b5adc7a947" containerName="cleanup"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275423    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff78d409-13d1-4c4c-b939-962ae9d46d63" containerName="payload-pre-processing"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275433    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfbaced6-9743-4b4d-b5bb-1c810ddd91a7" containerName="maas-api"
Sep 19 09:52:14.276049 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.275444    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2046ef60-4f0c-48ed-b65b-32c840fafe27" containerName="payload-pre-processing"
Sep 19 09:52:14.278487 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.278466    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.280811 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.280786    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ec0fc460-kserve-self-signed-certs\""
Sep 19 09:52:14.286530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.286503    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"]
Sep 19 09:52:14.301342 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.301313    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:52:14.358669 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.358629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-tmp-dir\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.358813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.358674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-home\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.358813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.358707    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-dshm\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.358813 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.358798    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-model-cache\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.358929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.358835    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc491dff-8a41-4f09-abe0-21c78faedf79-tls-certs\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.358929 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.358858    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l5gw\" (UniqueName: \"kubernetes.io/projected/cc491dff-8a41-4f09-abe0-21c78faedf79-kube-api-access-7l5gw\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460126    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-model-cache\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460170 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc491dff-8a41-4f09-abe0-21c78faedf79-tls-certs\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l5gw\" (UniqueName: \"kubernetes.io/projected/cc491dff-8a41-4f09-abe0-21c78faedf79-kube-api-access-7l5gw\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460228    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-tmp-dir\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-home\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460454 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-dshm\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460628    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-model-cache\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460675 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460660    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-tmp-dir\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.460790 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.460747    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-home\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.462694 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.462668    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-dshm\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.462935 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.462918    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc491dff-8a41-4f09-abe0-21c78faedf79-tls-certs\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.467971 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.467950    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l5gw\" (UniqueName: \"kubernetes.io/projected/cc491dff-8a41-4f09-abe0-21c78faedf79-kube-api-access-7l5gw\") pod \"e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") " pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.588213 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.588117    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:14.859996 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.859949    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"]
Sep 19 09:52:14.862220 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:52:14.862188    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc491dff_8a41_4f09_abe0_21c78faedf79.slice/crio-bd5103e61cc8dc835f354b2f6c069012fb2847e597d724d757aea0afb93085b7 WatchSource:0}: Error finding container bd5103e61cc8dc835f354b2f6c069012fb2847e597d724d757aea0afb93085b7: Status 404 returned error can't find the container with id bd5103e61cc8dc835f354b2f6c069012fb2847e597d724d757aea0afb93085b7
Sep 19 09:52:14.897912 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:14.897878    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28" event={"ID":"cc491dff-8a41-4f09-abe0-21c78faedf79","Type":"ContainerStarted","Data":"bd5103e61cc8dc835f354b2f6c069012fb2847e597d724d757aea0afb93085b7"}
Sep 19 09:52:15.904906 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:15.904865    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28" event={"ID":"cc491dff-8a41-4f09-abe0-21c78faedf79","Type":"ContainerStarted","Data":"7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da"}
Sep 19 09:52:15.924875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:15.924820    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28" podStartSLOduration=1.92480308 podStartE2EDuration="1.92480308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:15.922782526 +0000 UTC m=+2749.609587824" watchObservedRunningTime="2026-09-19 09:52:15.92480308 +0000 UTC m=+2749.611608377"
Sep 19 09:52:16.784022 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:16.783987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:52:24.588656 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:24.588605    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:24.588656 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:24.588666    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:24.600964 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:24.600941    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:24.956758 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:24.956725    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:52:26.068239 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.068206    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"]
Sep 19 09:52:26.068626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.068495    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy"
Sep 19 09:52:26.068626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.068505    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy"
Sep 19 09:52:26.068626 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.068582    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2d0f487-e80f-42d7-bfdf-ce63c922bec4" containerName="istio-proxy"
Sep 19 09:52:26.071760 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.071737    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.074330 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.074306    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ec0fc460-kserve-self-signed-certs\""
Sep 19 09:52:26.080877 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.080854    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"]
Sep 19 09:52:26.100327 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.100293    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:52:26.158390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.158352    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tmp-dir\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.158390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.158395    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tls-certs\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.158654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.158508    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-model-cache\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.158654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.158561    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-home\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.158654 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.158607    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hh6s\" (UniqueName: \"kubernetes.io/projected/2648f87a-9434-45a5-a466-9dd93b8a3a3f-kube-api-access-8hh6s\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.158763 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.158667    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-dshm\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.259995    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-model-cache\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260033 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-home\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hh6s\" (UniqueName: \"kubernetes.io/projected/2648f87a-9434-45a5-a466-9dd93b8a3a3f-kube-api-access-8hh6s\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-dshm\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260151    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tmp-dir\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260315 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260167    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tls-certs\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260530 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260512    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-model-cache\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260538    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-home\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.260591 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.260580    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tmp-dir\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.262789 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.262763    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-dshm\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.262981 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.262963    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tls-certs\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.268954 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.268928    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hh6s\" (UniqueName: \"kubernetes.io/projected/2648f87a-9434-45a5-a466-9dd93b8a3a3f-kube-api-access-8hh6s\") pod \"e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") " pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.381885 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.381846    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:26.653665 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.653507    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"]
Sep 19 09:52:26.656359 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:52:26.656323    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2648f87a_9434_45a5_a466_9dd93b8a3a3f.slice/crio-2a1f319356bb2cbff27e743d7b105cc10df28881059558c941cf19927f6874fa WatchSource:0}: Error finding container 2a1f319356bb2cbff27e743d7b105cc10df28881059558c941cf19927f6874fa: Status 404 returned error can't find the container with id 2a1f319356bb2cbff27e743d7b105cc10df28881059558c941cf19927f6874fa
Sep 19 09:52:26.955885 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.955793    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd" event={"ID":"2648f87a-9434-45a5-a466-9dd93b8a3a3f","Type":"ContainerStarted","Data":"78436080febf03dc9b913597b6435cfb4879ddc98abd5413ac0a4e85b90bd2a9"}
Sep 19 09:52:26.955885 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.955834    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd" event={"ID":"2648f87a-9434-45a5-a466-9dd93b8a3a3f","Type":"ContainerStarted","Data":"2a1f319356bb2cbff27e743d7b105cc10df28881059558c941cf19927f6874fa"}
Sep 19 09:52:26.972346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:26.972294    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd" podStartSLOduration=0.972276566 podStartE2EDuration="972.276566ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:26.971027562 +0000 UTC m=+2760.657832860" watchObservedRunningTime="2026-09-19 09:52:26.972276566 +0000 UTC m=+2760.659081864"
Sep 19 09:52:29.276154 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:29.276121    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:52:36.382300 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:36.382253    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:36.382300 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:36.382309    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:36.395135 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:36.395084    2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:52:37.013507 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:52:37.013478    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:53:02.419674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:02.419584    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"]
Sep 19 09:53:02.450220 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:02.450183    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:53:04.060128 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:04.060072    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"]
Sep 19 09:53:04.086617 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:04.086583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:53:04.466978 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:04.466946    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:53:13.259201 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:13.259165    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:53:17.439532 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.439474    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28" podUID="cc491dff-8a41-4f09-abe0-21c78faedf79" containerName="main" containerID="cri-o://7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da" gracePeriod=45
Sep 19 09:53:17.692864 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.692802    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:53:17.803840 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.803805    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc491dff-8a41-4f09-abe0-21c78faedf79-tls-certs\") pod \"cc491dff-8a41-4f09-abe0-21c78faedf79\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") "
Sep 19 09:53:17.804030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.803850    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-home\") pod \"cc491dff-8a41-4f09-abe0-21c78faedf79\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") "
Sep 19 09:53:17.804030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.803867    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-dshm\") pod \"cc491dff-8a41-4f09-abe0-21c78faedf79\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") "
Sep 19 09:53:17.804030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.803886    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-model-cache\") pod \"cc491dff-8a41-4f09-abe0-21c78faedf79\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") "
Sep 19 09:53:17.804030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.803908    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-tmp-dir\") pod \"cc491dff-8a41-4f09-abe0-21c78faedf79\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") "
Sep 19 09:53:17.804030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.804007    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l5gw\" (UniqueName: \"kubernetes.io/projected/cc491dff-8a41-4f09-abe0-21c78faedf79-kube-api-access-7l5gw\") pod \"cc491dff-8a41-4f09-abe0-21c78faedf79\" (UID: \"cc491dff-8a41-4f09-abe0-21c78faedf79\") "
Sep 19 09:53:17.804346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.804208    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc491dff-8a41-4f09-abe0-21c78faedf79" (UID: "cc491dff-8a41-4f09-abe0-21c78faedf79"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:17.804346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.804219    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-home" (OuterVolumeSpecName: "home") pod "cc491dff-8a41-4f09-abe0-21c78faedf79" (UID: "cc491dff-8a41-4f09-abe0-21c78faedf79"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:17.804346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.804232    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc491dff-8a41-4f09-abe0-21c78faedf79" (UID: "cc491dff-8a41-4f09-abe0-21c78faedf79"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:17.804346 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.804341    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:17.804520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.804354    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:17.804520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.804366    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:17.806420 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.806394    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc491dff-8a41-4f09-abe0-21c78faedf79-kube-api-access-7l5gw" (OuterVolumeSpecName: "kube-api-access-7l5gw") pod "cc491dff-8a41-4f09-abe0-21c78faedf79" (UID: "cc491dff-8a41-4f09-abe0-21c78faedf79"). InnerVolumeSpecName "kube-api-access-7l5gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:53:17.806498 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.806422    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-dshm" (OuterVolumeSpecName: "dshm") pod "cc491dff-8a41-4f09-abe0-21c78faedf79" (UID: "cc491dff-8a41-4f09-abe0-21c78faedf79"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:17.806498 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.806476    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc491dff-8a41-4f09-abe0-21c78faedf79-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc491dff-8a41-4f09-abe0-21c78faedf79" (UID: "cc491dff-8a41-4f09-abe0-21c78faedf79"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:53:17.905435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.905396    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l5gw\" (UniqueName: \"kubernetes.io/projected/cc491dff-8a41-4f09-abe0-21c78faedf79-kube-api-access-7l5gw\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:17.905435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.905427    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc491dff-8a41-4f09-abe0-21c78faedf79-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:17.905435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:17.905438    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc491dff-8a41-4f09-abe0-21c78faedf79-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:18.201702 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.201668    2581 generic.go:356] "Generic (PLEG): container finished" podID="cc491dff-8a41-4f09-abe0-21c78faedf79" containerID="7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da" exitCode=0
Sep 19 09:53:18.201873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.201714    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"
Sep 19 09:53:18.201873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.201745    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28" event={"ID":"cc491dff-8a41-4f09-abe0-21c78faedf79","Type":"ContainerDied","Data":"7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da"}
Sep 19 09:53:18.201873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.201786    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28" event={"ID":"cc491dff-8a41-4f09-abe0-21c78faedf79","Type":"ContainerDied","Data":"bd5103e61cc8dc835f354b2f6c069012fb2847e597d724d757aea0afb93085b7"}
Sep 19 09:53:18.201873 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.201803    2581 scope.go:117] "RemoveContainer" containerID="7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da"
Sep 19 09:53:18.210898 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.210879    2581 scope.go:117] "RemoveContainer" containerID="7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da"
Sep 19 09:53:18.211288 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:53:18.211264    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da\": container with ID starting with 7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da not found: ID does not exist" containerID="7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da"
Sep 19 09:53:18.211349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.211300    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da"} err="failed to get container status \"7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da\": rpc error: code = NotFound desc = could not find container \"7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da\": container with ID starting with 7bdb585ab330fc140bb1f4803964793b5742dc739720c1aa61c612f11f19d5da not found: ID does not exist"
Sep 19 09:53:18.222523 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.222484    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"]
Sep 19 09:53:18.226435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.226404    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ec0fc460-kserve-7464d98c79-slj28"]
Sep 19 09:53:18.830910 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:18.830865    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc491dff-8a41-4f09-abe0-21c78faedf79" path="/var/lib/kubelet/pods/cc491dff-8a41-4f09-abe0-21c78faedf79/volumes"
Sep 19 09:53:19.079887 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.079846    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd" podUID="2648f87a-9434-45a5-a466-9dd93b8a3a3f" containerName="main" containerID="cri-o://78436080febf03dc9b913597b6435cfb4879ddc98abd5413ac0a4e85b90bd2a9" gracePeriod=45
Sep 19 09:53:19.209208 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.209173    2581 generic.go:356] "Generic (PLEG): container finished" podID="2648f87a-9434-45a5-a466-9dd93b8a3a3f" containerID="78436080febf03dc9b913597b6435cfb4879ddc98abd5413ac0a4e85b90bd2a9" exitCode=0
Sep 19 09:53:19.209363 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.209246    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd" event={"ID":"2648f87a-9434-45a5-a466-9dd93b8a3a3f","Type":"ContainerDied","Data":"78436080febf03dc9b913597b6435cfb4879ddc98abd5413ac0a4e85b90bd2a9"}
Sep 19 09:53:19.336221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.336158    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:53:19.421126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421066    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tls-certs\") pod \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") "
Sep 19 09:53:19.421126 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421133    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-model-cache\") pod \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") "
Sep 19 09:53:19.421374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421187    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-home\") pod \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") "
Sep 19 09:53:19.421374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421206    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tmp-dir\") pod \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") "
Sep 19 09:53:19.421374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421235    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hh6s\" (UniqueName: \"kubernetes.io/projected/2648f87a-9434-45a5-a466-9dd93b8a3a3f-kube-api-access-8hh6s\") pod \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") "
Sep 19 09:53:19.421374 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421306    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-dshm\") pod \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\" (UID: \"2648f87a-9434-45a5-a466-9dd93b8a3a3f\") "
Sep 19 09:53:19.421599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421404    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "2648f87a-9434-45a5-a466-9dd93b8a3a3f" (UID: "2648f87a-9434-45a5-a466-9dd93b8a3a3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:19.421599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421421    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-home" (OuterVolumeSpecName: "home") pod "2648f87a-9434-45a5-a466-9dd93b8a3a3f" (UID: "2648f87a-9434-45a5-a466-9dd93b8a3a3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:19.421599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421462    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2648f87a-9434-45a5-a466-9dd93b8a3a3f" (UID: "2648f87a-9434-45a5-a466-9dd93b8a3a3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:19.421599 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421592    2581 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-home\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:19.421742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421610    2581 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tmp-dir\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:19.421742 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.421624    2581 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-model-cache\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:19.423524 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.423499    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2648f87a-9434-45a5-a466-9dd93b8a3a3f" (UID: "2648f87a-9434-45a5-a466-9dd93b8a3a3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:53:19.423624 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.423546    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2648f87a-9434-45a5-a466-9dd93b8a3a3f-kube-api-access-8hh6s" (OuterVolumeSpecName: "kube-api-access-8hh6s") pod "2648f87a-9434-45a5-a466-9dd93b8a3a3f" (UID: "2648f87a-9434-45a5-a466-9dd93b8a3a3f"). InnerVolumeSpecName "kube-api-access-8hh6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:53:19.423624 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.423580    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-dshm" (OuterVolumeSpecName: "dshm") pod "2648f87a-9434-45a5-a466-9dd93b8a3a3f" (UID: "2648f87a-9434-45a5-a466-9dd93b8a3a3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 09:53:19.522423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.522382    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hh6s\" (UniqueName: \"kubernetes.io/projected/2648f87a-9434-45a5-a466-9dd93b8a3a3f-kube-api-access-8hh6s\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:19.522423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.522417    2581 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2648f87a-9434-45a5-a466-9dd93b8a3a3f-dshm\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:19.522423 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:19.522427    2581 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2648f87a-9434-45a5-a466-9dd93b8a3a3f-tls-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:20.215403 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:20.215375    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"
Sep 19 09:53:20.215403 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:20.215394    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd" event={"ID":"2648f87a-9434-45a5-a466-9dd93b8a3a3f","Type":"ContainerDied","Data":"2a1f319356bb2cbff27e743d7b105cc10df28881059558c941cf19927f6874fa"}
Sep 19 09:53:20.215875 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:20.215437    2581 scope.go:117] "RemoveContainer" containerID="78436080febf03dc9b913597b6435cfb4879ddc98abd5413ac0a4e85b90bd2a9"
Sep 19 09:53:20.237577 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:20.237545    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"]
Sep 19 09:53:20.242249 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:20.242222    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ec0fc460-kserve-98d944795-6x8zd"]
Sep 19 09:53:20.473497 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:20.473412    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:53:20.830209 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:20.830130    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2648f87a-9434-45a5-a466-9dd93b8a3a3f" path="/var/lib/kubelet/pods/2648f87a-9434-45a5-a466-9dd93b8a3a3f/volumes"
Sep 19 09:53:26.674362 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:26.674327    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:53:27.262979 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.262937    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-rkd9k"]
Sep 19 09:53:27.263289 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.263274    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2648f87a-9434-45a5-a466-9dd93b8a3a3f" containerName="main"
Sep 19 09:53:27.263345 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.263291    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2648f87a-9434-45a5-a466-9dd93b8a3a3f" containerName="main"
Sep 19 09:53:27.263345 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.263308    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc491dff-8a41-4f09-abe0-21c78faedf79" containerName="main"
Sep 19 09:53:27.263345 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.263314    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc491dff-8a41-4f09-abe0-21c78faedf79" containerName="main"
Sep 19 09:53:27.263441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.263364    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2648f87a-9434-45a5-a466-9dd93b8a3a3f" containerName="main"
Sep 19 09:53:27.263441 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.263375    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc491dff-8a41-4f09-abe0-21c78faedf79" containerName="main"
Sep 19 09:53:27.266391 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.266370    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.268800 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.268778    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 09:53:27.268963 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.268840    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 09:53:27.269801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.269777    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 09:53:27.269906 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.269812    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 09:53:27.269906 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.269892    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzltl\""
Sep 19 09:53:27.276865 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.276842    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-rkd9k"]
Sep 19 09:53:27.392387 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.392353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55533662-d5c2-4f46-9b27-08d1d15024a7-openshift-service-ca\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.392514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.392414    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-metrics-certs\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.392514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.392437    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-625jn\" (UniqueName: \"kubernetes.io/projected/55533662-d5c2-4f46-9b27-08d1d15024a7-kube-api-access-625jn\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.392514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.392490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-maas-api-tls\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.495555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.494234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55533662-d5c2-4f46-9b27-08d1d15024a7-openshift-service-ca\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.495555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.494347    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-metrics-certs\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.495555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.494395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-625jn\" (UniqueName: \"kubernetes.io/projected/55533662-d5c2-4f46-9b27-08d1d15024a7-kube-api-access-625jn\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.495555 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.494488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-maas-api-tls\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.496188 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.496160    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55533662-d5c2-4f46-9b27-08d1d15024a7-openshift-service-ca\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.498549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.498518    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-metrics-certs\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.500688 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.500655    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-maas-api-tls\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.503154 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.503135    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-625jn\" (UniqueName: \"kubernetes.io/projected/55533662-d5c2-4f46-9b27-08d1d15024a7-kube-api-access-625jn\") pod \"maas-api-75876448df-rkd9k\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") " pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.577043 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.576944    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:27.852133 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.852090    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-rkd9k"]
Sep 19 09:53:27.854873 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:53:27.854849    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55533662_d5c2_4f46_9b27_08d1d15024a7.slice/crio-ec5bb0d295114d15fb1d16b3c1efa65ee99f09dbeb11dbedcf1f84c2abf1f437 WatchSource:0}: Error finding container ec5bb0d295114d15fb1d16b3c1efa65ee99f09dbeb11dbedcf1f84c2abf1f437: Status 404 returned error can't find the container with id ec5bb0d295114d15fb1d16b3c1efa65ee99f09dbeb11dbedcf1f84c2abf1f437
Sep 19 09:53:27.856596 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:27.856581    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 09:53:28.254439 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:28.254406    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k" event={"ID":"55533662-d5c2-4f46-9b27-08d1d15024a7","Type":"ContainerStarted","Data":"ec5bb0d295114d15fb1d16b3c1efa65ee99f09dbeb11dbedcf1f84c2abf1f437"}
Sep 19 09:53:29.260416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:29.260375    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k" event={"ID":"55533662-d5c2-4f46-9b27-08d1d15024a7","Type":"ContainerStarted","Data":"6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba"}
Sep 19 09:53:29.260801 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:29.260454    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:29.278674 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:29.278622    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k" podStartSLOduration=1.86726937 podStartE2EDuration="2.27860626s" totalImagesPullingTime="411.33689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:29.277306377 +0000 UTC m=+2822.964111678" watchObservedRunningTime="2026-09-19 09:53:29.27860626 +0000 UTC m=+2822.965411571"
Sep 19 09:53:34.748777 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.748737    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-rkd9k"]
Sep 19 09:53:34.749327 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.748956    2581 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k" podUID="55533662-d5c2-4f46-9b27-08d1d15024a7" containerName="maas-api" containerID="cri-o://6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba" gracePeriod=30
Sep 19 09:53:34.754268 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.754244    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:34.782664 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.782629    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"]
Sep 19 09:53:34.788216 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.788184    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.794760 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.794731    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"]
Sep 19 09:53:34.858864 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.858834    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5thk\" (UniqueName: \"kubernetes.io/projected/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-kube-api-access-h5thk\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.858864 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.858869    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-openshift-service-ca\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.859054 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.858886    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-metrics-certs\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.859054 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.858998    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-maas-api-tls\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.862966 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.862938    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:53:34.960226 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.960193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-maas-api-tls\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.960416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.960261    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5thk\" (UniqueName: \"kubernetes.io/projected/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-kube-api-access-h5thk\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.960416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.960278    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-openshift-service-ca\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.960416 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.960298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-metrics-certs\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.961083 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.961044    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-openshift-service-ca\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.962995 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.962970    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-maas-api-tls\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.963121 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.963032    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-metrics-certs\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.968588 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.968554    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5thk\" (UniqueName: \"kubernetes.io/projected/efd9a44c-bd1a-4850-a2a0-9ce55de58d0a-kube-api-access-h5thk\") pod \"maas-api-67f76686b9-9n6wv\" (UID: \"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a\") " pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:34.998520 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:34.998488    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:35.060776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.060681    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-maas-api-tls\") pod \"55533662-d5c2-4f46-9b27-08d1d15024a7\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") "
Sep 19 09:53:35.060776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.060726    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55533662-d5c2-4f46-9b27-08d1d15024a7-openshift-service-ca\") pod \"55533662-d5c2-4f46-9b27-08d1d15024a7\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") "
Sep 19 09:53:35.060776 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.060771    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-metrics-certs\") pod \"55533662-d5c2-4f46-9b27-08d1d15024a7\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") "
Sep 19 09:53:35.061030 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.060799    2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-625jn\" (UniqueName: \"kubernetes.io/projected/55533662-d5c2-4f46-9b27-08d1d15024a7-kube-api-access-625jn\") pod \"55533662-d5c2-4f46-9b27-08d1d15024a7\" (UID: \"55533662-d5c2-4f46-9b27-08d1d15024a7\") "
Sep 19 09:53:35.061236 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.061210    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55533662-d5c2-4f46-9b27-08d1d15024a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "55533662-d5c2-4f46-9b27-08d1d15024a7" (UID: "55533662-d5c2-4f46-9b27-08d1d15024a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 09:53:35.063034 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.063007    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "55533662-d5c2-4f46-9b27-08d1d15024a7" (UID: "55533662-d5c2-4f46-9b27-08d1d15024a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:53:35.063171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.063075    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "55533662-d5c2-4f46-9b27-08d1d15024a7" (UID: "55533662-d5c2-4f46-9b27-08d1d15024a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 09:53:35.063171 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.063143    2581 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55533662-d5c2-4f46-9b27-08d1d15024a7-kube-api-access-625jn" (OuterVolumeSpecName: "kube-api-access-625jn") pod "55533662-d5c2-4f46-9b27-08d1d15024a7" (UID: "55533662-d5c2-4f46-9b27-08d1d15024a7"). InnerVolumeSpecName "kube-api-access-625jn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 09:53:35.125546 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.125494    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:35.161957 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.161920    2581 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-maas-api-tls\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:35.161957 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.161951    2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55533662-d5c2-4f46-9b27-08d1d15024a7-openshift-service-ca\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:35.161957 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.161963    2581 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55533662-d5c2-4f46-9b27-08d1d15024a7-metrics-certs\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:35.162221 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.161973    2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-625jn\" (UniqueName: \"kubernetes.io/projected/55533662-d5c2-4f46-9b27-08d1d15024a7-kube-api-access-625jn\") on node \"ip-10-0-131-218.ec2.internal\" DevicePath \"\""
Sep 19 09:53:35.290056 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.290023    2581 generic.go:356] "Generic (PLEG): container finished" podID="55533662-d5c2-4f46-9b27-08d1d15024a7" containerID="6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba" exitCode=0
Sep 19 09:53:35.290266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.290065    2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k"
Sep 19 09:53:35.290266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.290134    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k" event={"ID":"55533662-d5c2-4f46-9b27-08d1d15024a7","Type":"ContainerDied","Data":"6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba"}
Sep 19 09:53:35.290266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.290195    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75876448df-rkd9k" event={"ID":"55533662-d5c2-4f46-9b27-08d1d15024a7","Type":"ContainerDied","Data":"ec5bb0d295114d15fb1d16b3c1efa65ee99f09dbeb11dbedcf1f84c2abf1f437"}
Sep 19 09:53:35.290266 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.290224    2581 scope.go:117] "RemoveContainer" containerID="6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba"
Sep 19 09:53:35.299876 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.299860    2581 scope.go:117] "RemoveContainer" containerID="6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba"
Sep 19 09:53:35.300197 ip-10-0-131-218 kubenswrapper[2581]: E0919 09:53:35.300178    2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba\": container with ID starting with 6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba not found: ID does not exist" containerID="6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba"
Sep 19 09:53:35.300258 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.300206    2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba"} err="failed to get container status \"6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba\": rpc error: code = NotFound desc = could not find container \"6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba\": container with ID starting with 6c8e5bbb087ed3252e2f737c55b61ec204104a201b936591a4a89afb13da29ba not found: ID does not exist"
Sep 19 09:53:35.311891 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.311834    2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-rkd9k"]
Sep 19 09:53:35.316426 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.316403    2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-75876448df-rkd9k"]
Sep 19 09:53:35.398892 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:35.398867    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"]
Sep 19 09:53:35.400997 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:53:35.400972    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd9a44c_bd1a_4850_a2a0_9ce55de58d0a.slice/crio-335f1ffaa97df335ed0baa4e36022e7b46b9d9462d9195022d66abaeb25760ea WatchSource:0}: Error finding container 335f1ffaa97df335ed0baa4e36022e7b46b9d9462d9195022d66abaeb25760ea: Status 404 returned error can't find the container with id 335f1ffaa97df335ed0baa4e36022e7b46b9d9462d9195022d66abaeb25760ea
Sep 19 09:53:36.297486 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:36.297449    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv" event={"ID":"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a","Type":"ContainerStarted","Data":"84cee2f7fc542aa045780fbd6e284912ed191ab43bac750e57f9aa8f85337415"}
Sep 19 09:53:36.297926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:36.297502    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv" event={"ID":"efd9a44c-bd1a-4850-a2a0-9ce55de58d0a","Type":"ContainerStarted","Data":"335f1ffaa97df335ed0baa4e36022e7b46b9d9462d9195022d66abaeb25760ea"}
Sep 19 09:53:36.297926 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:36.297543    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:36.314953 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:36.314896    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv" podStartSLOduration=1.890123112 podStartE2EDuration="2.314880776s" totalImagesPullingTime="424.757664ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:36.313214463 +0000 UTC m=+2830.000019763" watchObservedRunningTime="2026-09-19 09:53:36.314880776 +0000 UTC m=+2830.001686104"
Sep 19 09:53:36.830975 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:36.830938    2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55533662-d5c2-4f46-9b27-08d1d15024a7" path="/var/lib/kubelet/pods/55533662-d5c2-4f46-9b27-08d1d15024a7/volumes"
Sep 19 09:53:41.369023 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:41.368987    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:53:42.307890 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:42.307861    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67f76686b9-9n6wv"
Sep 19 09:53:47.067611 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:47.067574    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:53:53.772279 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:53:53.772240    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-cxbrk"]
Sep 19 09:54:03.103781 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:03.103752    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-98ss7_c8b98810-667c-4ad9-8724-7c64dd6be296/copy-manifests/0.log"
Sep 19 09:54:03.114302 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:03.114270    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-98ss7_c8b98810-667c-4ad9-8724-7c64dd6be296/manager/0.log"
Sep 19 09:54:03.572141 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:03.572094    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-67f76686b9-9n6wv_efd9a44c-bd1a-4850-a2a0-9ce55de58d0a/maas-api/0.log"
Sep 19 09:54:05.117444 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:05.117412    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-6td5q_94889a2a-5c9f-4ede-9684-975df34d6d91/kuadrant-console-plugin/0.log"
Sep 19 09:54:05.353747 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:05.353695    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-cxbrk_54b5ae5b-3d7b-46b8-ab65-af70cf445de4/limitador/0.log"
Sep 19 09:54:06.158204 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:06.158162    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-m64x8_d0a03e76-206c-493a-94ab-980d8926fba5/kube-auth-proxy/0.log"
Sep 19 09:54:07.283321 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:07.283264    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-bjpd5_b9e7a2b5-8640-421a-b79e-451a01510c01/main/0.log"
Sep 19 09:54:07.500195 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:07.500168    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-bmwg5_91a60185-6e3e-434c-9001-f6e09e21c7f2/main/0.log"
Sep 19 09:54:07.611667 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:07.611581    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-55bsw_ba6c19fa-1d9f-4e21-9144-dd12f22b25f9/main/0.log"
Sep 19 09:54:12.804831 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:12.804793    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:54:12.880661 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:12.880631    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:54:15.608390 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:15.608353    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kr42v_60bb9f92-155a-4f69-b9d8-845edce54d11/global-pull-secret-syncer/0.log"
Sep 19 09:54:15.729736 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:15.729701    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log"
Sep 19 09:54:15.868987 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:15.868897    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-218.ec2.internal_b7e4c44a51bdfd73ff16543061b2fc92/haproxy/0.log"
Sep 19 09:54:20.315797 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:20.315726    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-6td5q_94889a2a-5c9f-4ede-9684-975df34d6d91/kuadrant-console-plugin/0.log"
Sep 19 09:54:20.402349 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:20.402321    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-cxbrk_54b5ae5b-3d7b-46b8-ab65-af70cf445de4/limitador/0.log"
Sep 19 09:54:22.233757 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:22.233719    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-98ptl_6e7c3e3d-26d5-4268-88fa-70e24f64d7ed/cluster-monitoring-operator/0.log"
Sep 19 09:54:22.549334 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:22.549258    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9xtr6_2325259e-d06d-473c-a97f-d647ece2128d/node-exporter/0.log"
Sep 19 09:54:22.571919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:22.571889    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9xtr6_2325259e-d06d-473c-a97f-d647ece2128d/kube-rbac-proxy/0.log"
Sep 19 09:54:22.593919 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:22.593892    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9xtr6_2325259e-d06d-473c-a97f-d647ece2128d/init-textfile/0.log"
Sep 19 09:54:24.790858 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:24.790830    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-k95hw_a2870d9c-99a3-4533-a2d7-9fe87b230b6b/networking-console-plugin/0.log"
Sep 19 09:54:25.906906 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.906860    2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"]
Sep 19 09:54:25.907630 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.907607    2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55533662-d5c2-4f46-9b27-08d1d15024a7" containerName="maas-api"
Sep 19 09:54:25.907723 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.907640    2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="55533662-d5c2-4f46-9b27-08d1d15024a7" containerName="maas-api"
Sep 19 09:54:25.907797 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.907778    2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="55533662-d5c2-4f46-9b27-08d1d15024a7" containerName="maas-api"
Sep 19 09:54:25.912054 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.912027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:25.917549 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.917525    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2bwqb\"/\"kube-root-ca.crt\""
Sep 19 09:54:25.917705 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.917535    2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2bwqb\"/\"default-dockercfg-5g8bj\""
Sep 19 09:54:25.917828 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.917808    2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2bwqb\"/\"openshift-service-ca.crt\""
Sep 19 09:54:25.927472 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.927447    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"]
Sep 19 09:54:25.958455 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.958409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6hht\" (UniqueName: \"kubernetes.io/projected/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-kube-api-access-n6hht\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:25.958636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.958466    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-sys\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:25.958636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.958568    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-lib-modules\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:25.958636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.958605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-proc\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:25.958636 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:25.958629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-podres\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.059692 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059657    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6hht\" (UniqueName: \"kubernetes.io/projected/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-kube-api-access-n6hht\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.059692 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-sys\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.059916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059718    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-lib-modules\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.059916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059809    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-sys\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.059916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059841    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-lib-modules\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.059916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059844    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-proc\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.059916 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059881    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-podres\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.060130 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.059942    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-proc\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.060130 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.060054    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-podres\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.069435 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.069410    2581 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6hht\" (UniqueName: \"kubernetes.io/projected/ade7dbdb-aebf-457d-b31d-3c8f81eaabe5-kube-api-access-n6hht\") pod \"perf-node-gather-daemonset-7j9vm\" (UID: \"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5\") " pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.221857 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.221763    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:26.494514 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.494483    2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"]
Sep 19 09:54:26.497262 ip-10-0-131-218 kubenswrapper[2581]: W0919 09:54:26.497230    2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podade7dbdb_aebf_457d_b31d_3c8f81eaabe5.slice/crio-9ffbfff5b252c1414c39c623d4c7ce5fd54e3338770f62686eb6985a492831d7 WatchSource:0}: Error finding container 9ffbfff5b252c1414c39c623d4c7ce5fd54e3338770f62686eb6985a492831d7: Status 404 returned error can't find the container with id 9ffbfff5b252c1414c39c623d4c7ce5fd54e3338770f62686eb6985a492831d7
Sep 19 09:54:26.535160 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.535129    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm" event={"ID":"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5","Type":"ContainerStarted","Data":"9ffbfff5b252c1414c39c623d4c7ce5fd54e3338770f62686eb6985a492831d7"}
Sep 19 09:54:26.792677 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:26.792589    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-jtvdj_cd8bb87f-3e17-4520-924c-6f60156337fc/volume-data-source-validator/0.log"
Sep 19 09:54:27.541642 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:27.541606    2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm" event={"ID":"ade7dbdb-aebf-457d-b31d-3c8f81eaabe5","Type":"ContainerStarted","Data":"97a671f9311a2dba16e546cd5e7776c99c13923babeb02483db5de1e18cf8144"}
Sep 19 09:54:27.542029 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:27.541720    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:27.557370 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:27.557320    2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm" podStartSLOduration=2.557304914 podStartE2EDuration="2.557304914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:54:27.556522429 +0000 UTC m=+2881.243327724" watchObservedRunningTime="2026-09-19 09:54:27.557304914 +0000 UTC m=+2881.244110215"
Sep 19 09:54:27.919065 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:27.919028    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d7jp4_0a06e1ca-d6f0-4504-bef6-b573918caa98/dns/0.log"
Sep 19 09:54:27.938461 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:27.938433    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-d7jp4_0a06e1ca-d6f0-4504-bef6-b573918caa98/kube-rbac-proxy/0.log"
Sep 19 09:54:28.048566 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:28.048541    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4g6kn_c80ec8c6-a38b-496c-baaa-33d918287237/dns-node-resolver/0.log"
Sep 19 09:54:28.681427 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:28.681381    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jjlx7_10fdafc3-1641-4fa0-8ec1-3d403c30e9b5/node-ca/0.log"
Sep 19 09:54:30.119008 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:30.118977    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-m64x8_d0a03e76-206c-493a-94ab-980d8926fba5/kube-auth-proxy/0.log"
Sep 19 09:54:31.254410 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:31.254378    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b5thm_c0ad2ff9-8673-44cd-989f-893a35ed9c59/serve-healthcheck-canary/0.log"
Sep 19 09:54:31.758164 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:31.758081    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-v684g_4962a821-49df-4abb-ae80-375b5d2724fd/insights-operator/0.log"
Sep 19 09:54:31.759899 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:31.759874    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-v684g_4962a821-49df-4abb-ae80-375b5d2724fd/insights-operator/1.log"
Sep 19 09:54:31.854034 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:31.853986    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7m7zt_c83865a1-9ca7-46ec-9edf-75a9899f261a/kube-rbac-proxy/0.log"
Sep 19 09:54:31.873836 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:31.873803    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7m7zt_c83865a1-9ca7-46ec-9edf-75a9899f261a/exporter/0.log"
Sep 19 09:54:31.895059 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:31.895034    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7m7zt_c83865a1-9ca7-46ec-9edf-75a9899f261a/extractor/0.log"
Sep 19 09:54:32.270072 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:32.270037    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:54:32.330640 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:32.330604    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-592rd_6ecfb471-1cc3-414d-add4-29f6ce623c8c/konnectivity-agent/0.log" limit=1024
Sep 19 09:54:33.556955 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:33.556926    2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2bwqb/perf-node-gather-daemonset-7j9vm"
Sep 19 09:54:34.727429 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:34.727393    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-98ss7_c8b98810-667c-4ad9-8724-7c64dd6be296/manager/0.log"
Sep 19 09:54:34.745107 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:34.745081    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56479975c9-98ss7_c8b98810-667c-4ad9-8724-7c64dd6be296/copy-manifests/0.log"
Sep 19 09:54:36.176877 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:36.176846    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-m64x8_d0a03e76-206c-493a-94ab-980d8926fba5/kube-auth-proxy/0.log"
Sep 19 09:54:36.209799 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:36.209774    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-9b48b7f58-nmwch_d382c572-f402-4612-9ea9-6a391d2ddb66/manager/0.log"
Sep 19 09:54:38.271855 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:38.271824    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d645f99d7-m64x8_d0a03e76-206c-493a-94ab-980d8926fba5/kube-auth-proxy/0.log"
Sep 19 09:54:41.408645 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:41.408611    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-ms6vs_35e70a78-17a8-4f84-9e76-561b60cbb75e/migrator/0.log"
Sep 19 09:54:41.442161 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:41.442130    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-ms6vs_35e70a78-17a8-4f84-9e76-561b60cbb75e/graceful-termination/0.log"
Sep 19 09:54:41.782429 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:41.782320    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-p2m8b_3bc99bcb-8487-4ae8-89e6-de3272b082f1/kube-storage-version-migrator-operator/1.log"
Sep 19 09:54:41.784091 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:41.784057    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-p2m8b_3bc99bcb-8487-4ae8-89e6-de3272b082f1/kube-storage-version-migrator-operator/0.log"
Sep 19 09:54:42.888991 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:42.888962    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hc2pd_173d481c-5843-462c-b6fc-b645554ea972/kube-multus-additional-cni-plugins/0.log"
Sep 19 09:54:42.909014 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:42.908981    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hc2pd_173d481c-5843-462c-b6fc-b645554ea972/egress-router-binary-copy/0.log"
Sep 19 09:54:42.930509 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:42.930480    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hc2pd_173d481c-5843-462c-b6fc-b645554ea972/cni-plugins/0.log"
Sep 19 09:54:42.948330 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:42.948302    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hc2pd_173d481c-5843-462c-b6fc-b645554ea972/bond-cni-plugin/0.log"
Sep 19 09:54:42.967787 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:42.967754    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hc2pd_173d481c-5843-462c-b6fc-b645554ea972/routeoverride-cni/0.log"
Sep 19 09:54:42.986904 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:42.986863    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hc2pd_173d481c-5843-462c-b6fc-b645554ea972/whereabouts-cni-bincopy/0.log"
Sep 19 09:54:43.004959 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:43.004932    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hc2pd_173d481c-5843-462c-b6fc-b645554ea972/whereabouts-cni/0.log"
Sep 19 09:54:43.244765 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:43.244680    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-g6pzw_0ddc969e-c89d-4345-a7a5-d901f9ebe551/kube-multus/0.log"
Sep 19 09:54:43.270533 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:43.270474    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-485pv_e0c101dc-58b0-481d-9859-234a79696788/network-metrics-daemon/0.log"
Sep 19 09:54:43.284261 ip-10-0-131-218 kubenswrapper[2581]: I0919 09:54:43.284219    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-485pv_e0c101dc-58b0-481d-9859-234a79696788/kube-rbac-proxy/0.log"