Sep 10 22:03:20.680937 ip-10-0-137-25 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 22:03:20.680949 ip-10-0-137-25 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 22:03:20.680956 ip-10-0-137-25 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 22:03:20.681218 ip-10-0-137-25 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 22:03:30.786291 ip-10-0-137-25 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 22:03:30.786309 ip-10-0-137-25 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 03034f671e5a4368b8a596834b10d786 --
Sep 10 22:05:30.502747 ip-10-0-137-25 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 22:05:30.883777 ip-10-0-137-25 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 22:05:30.883777 ip-10-0-137-25 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 22:05:30.883777 ip-10-0-137-25 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 22:05:30.883777 ip-10-0-137-25 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 22:05:30.883777 ip-10-0-137-25 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 22:05:30.885922 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.885841    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 22:05:30.890691 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890675    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:05:30.890691 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890690    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890694    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890697    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890700    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890704    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890707    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890710    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890712    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890715    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890718    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890721    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890724    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890727    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890729    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890732    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890739    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890742    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890744    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890747    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:05:30.890759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890750    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890753    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890756    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890758    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890762    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890764    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890767    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890770    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890773    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890776    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890778    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890781    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890783    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890786    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890789    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890791    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890795    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890797    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890800    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890803    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:05:30.891216 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890805    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890808    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890810    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890812    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890815    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890817    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890820    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890822    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890825    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890828    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890831    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890833    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890836    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890839    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890842    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890844    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890847    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890850    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890853    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890855    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:05:30.891735 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890858    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890862    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890865    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890867    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890870    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890872    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890875    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890877    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890880    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890883    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890885    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890888    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890891    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890893    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890896    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890898    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890901    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890903    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890906    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890908    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:05:30.892225 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890912    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890917    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890922    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890926    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890929    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890932    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890935    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890938    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890941    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890944    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890947    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890950    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890953    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890956    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890958    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890961    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890963    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890967    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.890970    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:05:30.892789 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891702    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891711    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891714    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891718    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891722    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891725    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891728    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891730    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891733    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891736    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891740    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891742    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891746    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891748    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891751    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891754    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891757    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891760    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891762    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891765    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:05:30.893242 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891767    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891771    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891773    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891776    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891778    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891780    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891783    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891786    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891788    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891791    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891793    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891796    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891799    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891801    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891804    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891806    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891809    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891812    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891815    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891818    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:05:30.893851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891820    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891823    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891828    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891831    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891834    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891837    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891840    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891842    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891845    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891847    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891850    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891852    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891855    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891857    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891860    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891862    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891865    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891868    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891870    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:05:30.894348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891872    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891875    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891878    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891880    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891883    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891885    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891888    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891890    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891893    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891895    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891898    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891900    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891903    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891906    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891908    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891911    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891914    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891916    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891919    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891921    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:05:30.894837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891925    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891927    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891930    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891932    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891934    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891937    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891939    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891942    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891944    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891947    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891949    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891952    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891955    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891958    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891961    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891963    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891965    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891968    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891970    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.891973    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:05:30.895318 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892325    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892329    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892332    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892335    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892338    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892343    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892346    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892349    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892353    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892355    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892358    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892361    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892365    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892368    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892371    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892374    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892377    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892380    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892383    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:05:30.895837 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892385    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892388    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892390    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892393    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892395    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892398    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892400    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892403    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892405    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892408    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892410    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892413    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892415    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892419    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892421    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892423    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892426    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892429    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892432    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892449    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:05:30.896307 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892452    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892455    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892457    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892460    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892462    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892465    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892467    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892470    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892472    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892474    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892477    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892479    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892482    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892484    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892487    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892490    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892492    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892494    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892497    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892499    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:05:30.896804 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892502    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892504    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892508    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892511    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892513    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892516    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892519    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892521    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892524    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892527    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892529    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892532    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892537    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892539    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892541    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892544    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892546    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892548    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892551    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892553    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:05:30.897287 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892556    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892558    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892561    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892563    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892566    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892568    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892571    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892573    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892576    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892578    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892580    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892583    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892585    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892588    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892590    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892593    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892596    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892600    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892603    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.892605    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:05:30.897841 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892679    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892709    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892716    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892720    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892725    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892729    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892733    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892737    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892740    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892743    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892747    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892750    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892753    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892756    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892759    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892762    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892765    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892768    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892772    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892774    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892777    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892780    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892784    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892787    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 22:05:30.898347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892790    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892793    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892797    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892799    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892802    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892807    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892810    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892813    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892816    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892820    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892823    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892826    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892831    2563 flags.go:64] FLAG: --enable-server="true"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892834    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892838    2563 flags.go:64] FLAG: --event-burst="100"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892841    2563 flags.go:64] FLAG: --event-qps="50"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892844    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892847    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892850    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892854    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892857    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892860    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892863    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892866    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892869    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 22:05:30.898943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892871    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892874    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892877    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892880    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892883    2563 flags.go:64] FLAG: --feature-gates=""
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892886    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892889    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892892    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892895    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892898    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892901    2563 flags.go:64] FLAG: --help="false"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892904    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-137-25.ec2.internal"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892908    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892911    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892913    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892916    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892920    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892922    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892925    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892932    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892935    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892938    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892941    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892944    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 10 22:05:30.899597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892946    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892949    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892952    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892955    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892957    2563 flags.go:64] FLAG: --lock-file=""
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892961    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892964    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892967    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892972    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892975    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892978    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892980    2563 flags.go:64] FLAG: --logging-format="text"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892983    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892986    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892989    2563 flags.go:64] FLAG: --manifest-url=""
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892992    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892996    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.892999    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893002    2563 flags.go:64] FLAG: --max-pods="110"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893005    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893009    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893012    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893015    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893018    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893021    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 22:05:30.900182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893023    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893031    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893035    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893038    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893041    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893044    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893049    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893052    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893055    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893058    2563 flags.go:64] FLAG: --port="10250"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893061    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893064    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00c83a9cd9afb0a00"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893067    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893069    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893072    2563 flags.go:64] FLAG: --register-node="true"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893075    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893078    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893081    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893084    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893086    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893090    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893093    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893096    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893099    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893102    2563 flags.go:64] FLAG: --runonce="false"
Sep 10 22:05:30.900838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893104    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893107    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893110    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893114    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893116    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893119    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893122    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893125    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893128    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893132    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893135    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893138    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893141    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893144    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893147    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893152    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893155    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893158    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893161    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893164    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893166    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893169    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893172    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893175    2563 flags.go:64] FLAG: --v="2"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893178    2563 flags.go:64] FLAG: --version="false"
Sep 10 22:05:30.901535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893182    2563 flags.go:64] FLAG: --vmodule=""
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893186    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.893189    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893299    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893303    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893306    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893309    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893312    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893314    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893317    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893321    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893323    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893326    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893329    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893331    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893334    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893338    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893340    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893343    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893346    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:05:30.902407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893349    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893351    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893354    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893356    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893359    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893362    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893364    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893367    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893370    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893372    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893375    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893378    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893380    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893383    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893385    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893388    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893391    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893393    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893396    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893399    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:05:30.903217 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893401    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893404    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893407    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893410    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893413    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893415    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893418    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893421    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893424    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893428    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893430    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893451    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893454    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893457    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893460    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893462    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893465    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893468    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893470    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893473    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:05:30.903759 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893475    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893478    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893480    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893483    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893486    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893488    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893491    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893494    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893496    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893499    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893501    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893504    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893508    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893512    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893517    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893519    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893522    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893525    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893527    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:05:30.904266 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893530    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893534    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893537    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893541    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893544    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893547    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893550    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893553    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893557    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893560    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893563    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893566    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893569    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893572    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893575    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893578    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893580    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893583    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893586    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:05:30.904756 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893589    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893591    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893594    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.893596    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.894540    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.903128    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.903144    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.903164    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.903173    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903231    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903236    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903239    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903244    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903248    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:05:30.905223 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903251    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903254    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903257    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903260    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903263    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903267    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903270    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903273    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903276    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903279    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903281    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903284    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903286    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903289    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903292    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903295    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903297    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903300    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903303    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:05:30.905682 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903306    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903308    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903311    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903314    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903317    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903320    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903323    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903325    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903328    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903330    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903333    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903336    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903339    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903341    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903344    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903346    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903349    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903352    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903354    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903357    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:05:30.906213 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903359    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903362    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903366    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903369    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903372    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903374    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903377    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903379    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903382    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903384    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903387    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903390    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903393    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903395    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903398    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903401    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903403    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903407    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903410    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903412    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:05:30.906723 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903415    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903418    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903421    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903425    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903427    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903430    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903432    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903455    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903458    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903461    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903464    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903466    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903469    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903472    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903475    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903478    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903481    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903483    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903486    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903489    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:05:30.907226 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903491    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903494    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903496    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903499    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903501    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903504    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903506    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903510    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903512    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903515    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903518    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903520    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903523    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903526    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:05:30.907752 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903528    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.903534    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903651    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903657    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903660    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903663    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903667    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903671    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903674    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903677    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903680    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903682    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903685    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903688    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 22:05:30.908096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903691    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903693    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903696    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903698    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903701    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903703    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903706    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903708    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903711    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903714    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903716    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903719    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903722    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903725    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903727    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903730    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903732    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903735    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903737    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903740    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 22:05:30.908463 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903743    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903745    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903748    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903751    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903753    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903755    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903758    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903760    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903763    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903766    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903768    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903771    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903774    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903777    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903780    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903782    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903784    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903787    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903789    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 22:05:30.908954 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903792    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903794    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903797    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903799    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903802    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903805    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903807    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903810    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903812    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903815    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903818    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903820    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903823    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903825    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903827    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903831    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903834    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903836    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903839    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 22:05:30.909514 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903841    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903845    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903848    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903851    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903854    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903857    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903859    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903862    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903865    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903868    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903870    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903873    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903875    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903878    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903880    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903883    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903885    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903887    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903890    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 22:05:30.909984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903893    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903895    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903897    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903900    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903903    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903905    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903908    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903910    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903913    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:30.903915    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.903920    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.904027    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.905899    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.906633    2563 server.go:1019] "Starting client certificate rotation"
Sep 10 22:05:30.910457 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.906729    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 22:05:30.910810 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.906759    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 22:05:30.910810 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.909356    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 22:05:30.912552 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.912536    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 22:05:30.921414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.921394    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 10 22:05:30.925641 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.925626    2563 log.go:25] "Validated CRI v1 image API"
Sep 10 22:05:30.926500 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.926485    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 22:05:30.929962 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.929940    2563 fs.go:135] Filesystem UUIDs: map[1ba94968-9660-472c-b583-1bca5d2ccaf7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f8c71be1-62af-483c-897a-8ebd78888234:/dev/nvme0n1p4]
Sep 10 22:05:30.930031 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.929962    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 22:05:30.935925 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.935819    2563 manager.go:216] Machine: {Timestamp:2026-09-10 22:05:30.934552978 +0000 UTC m=+0.319058394 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3152978 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23a8a08027144624adb9e072018868 SystemUUID:ec23a8a0-8027-1446-24ad-b9e072018868 BootID:03034f67-1e5a-4368-b8a5-96834b10d786 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:ac:83:14:21 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:ac:83:14:21 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:64:a9:0b:4c:c9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 22:05:30.935925 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.935919    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 22:05:30.936064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.935995    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 22:05:30.936358 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.936343    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 22:05:30.938964 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.938943    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 22:05:30.939092 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.938969    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-25.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 22:05:30.940781 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.940769    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 22:05:30.940815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.940783    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 22:05:30.940815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.940791    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 22:05:30.941701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.941690    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 22:05:30.942840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.942830    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 22:05:30.945530 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.945518    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 22:05:30.945646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.945637    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 22:05:30.945679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.945666    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 22:05:30.945679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.945676    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 22:05:30.945737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.945689    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 22:05:30.945737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.945701    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 10 22:05:30.945737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.945709    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 22:05:30.946075 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.946055    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 22:05:30.946075 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.946074    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 22:05:30.949344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.949321    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 22:05:30.951685 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.951672    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 22:05:30.951731 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.951702    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 22:05:30.954180 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954167    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954186    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954192    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954197    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954206    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954211    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954224    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954229    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 22:05:30.954232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954235    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 22:05:30.954448 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954241    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 22:05:30.954448 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954249    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 22:05:30.954448 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.954254    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 22:05:30.957139 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.957118    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 22:05:30.957139 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.957123    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 22:05:30.957930 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.957918    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 22:05:30.957930 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.957930    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 22:05:30.961994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.961980    2563 server.go:1268] "Started kubelet"
Sep 10 22:05:30.962124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.962078    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 22:05:30.962124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.962095    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 22:05:30.962198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.962147    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 22:05:30.962806 ip-10-0-137-25 systemd[1]: Started Kubernetes Kubelet.
Sep 10 22:05:30.963915 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.963902    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 22:05:30.965149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.965135    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 22:05:30.966767 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.966752    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-25.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 22:05:30.970029 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.970004    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 22:05:30.970373 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.970357    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 22:05:30.970583 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.970571    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 22:05:30.970758 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.970703    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:30.971116 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971098    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 22:05:30.971116 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971100    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 22:05:30.971252 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971124    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 22:05:30.971252 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971200    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 22:05:30.971252 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971209    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 22:05:30.971576 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971501    2563 factory.go:55] Registering systemd factory
Sep 10 22:05:30.971644 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971611    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 10 22:05:30.972041 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.971856    2563 factory.go:153] Registering CRI-O factory
Sep 10 22:05:30.972127 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.972046    2563 factory.go:223] Registration of the crio container factory successfully
Sep 10 22:05:30.972127 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.972093    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 22:05:30.972127 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.972110    2563 factory.go:103] Registering Raw factory
Sep 10 22:05:30.972127 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.972125    2563 manager.go:1196] Started watching for new ooms in manager
Sep 10 22:05:30.973227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.973213    2563 manager.go:349] Starting recovery of all containers
Sep 10 22:05:30.975650 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.975628    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 22:05:30.975944 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.975916    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-25.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 22:05:30.976749 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.976727    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sp5pr"
Sep 10 22:05:30.976987 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.975667    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-25.ec2.internal.18d414e6367a63da  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-25.ec2.internal,UID:ip-10-0-137-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-25.ec2.internal,},FirstTimestamp:2026-09-10 22:05:30.961953754 +0000 UTC m=+0.346459170,LastTimestamp:2026-09-10 22:05:30.961953754 +0000 UTC m=+0.346459170,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-25.ec2.internal,}"
Sep 10 22:05:30.985144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.985117    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sp5pr"
Sep 10 22:05:30.987116 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.987099    2563 manager.go:354] Recovery completed
Sep 10 22:05:30.991604 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.991588    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:05:30.993323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.993306    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:05:30.993417 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.993335    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:05:30.993417 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.993345    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:05:30.993944 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.993930    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 22:05:30.993944 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.993943    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 22:05:30.994046 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.993957    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 22:05:30.994046 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.994005    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:05:30.994663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.994649    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:05:30.994722 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.994672    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:05:30.994722 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.994681    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:05:30.995079 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.995068    2563 policy_none.go:49] "None policy: Start"
Sep 10 22:05:30.995111 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.995084    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 22:05:30.995111 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.995093    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 22:05:30.995745 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:30.995673    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-25.ec2.internal.18d414e6385901dd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-25.ec2.internal,UID:ip-10-0-137-25.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-25.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-25.ec2.internal,},FirstTimestamp:2026-09-10 22:05:30.993320413 +0000 UTC m=+0.377825830,LastTimestamp:2026-09-10 22:05:30.993320413 +0000 UTC m=+0.377825830,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-25.ec2.internal,}"
Sep 10 22:05:30.996148 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:30.996137    2563 policy_none.go:47] "Start"
Sep 10 22:05:31.030079 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.030062    2563 manager.go:337] "Starting Device Plugin manager"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.030107    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.030120    2563 server.go:87] "Starting device plugin registration server"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.030493    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.030508    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.030720    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.030789    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.030796    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.031961    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 22:05:31.046617 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.032028    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.102562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.102533    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 22:05:31.103882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.103864    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 22:05:31.103975 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.103910    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 22:05:31.103975 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.103933    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 22:05:31.104061 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.103974    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 22:05:31.106196 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.106176    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:05:31.131390 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.131362    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:05:31.132148 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.132134    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:05:31.132206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.132161    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:05:31.132206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.132170    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:05:31.132206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.132190    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.138705 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.138648    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.138705 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.138667    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-25.ec2.internal\": node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.153545 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.153519    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.204486 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.204466    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal"]
Sep 10 22:05:31.204553 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.204524    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:05:31.205992 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.205978    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:05:31.206038 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.206009    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:05:31.206038 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.206025    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:05:31.207513 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.207502    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:05:31.207637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.207622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.207677 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.207650    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:05:31.208280 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.208262    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:05:31.208369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.208294    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:05:31.208369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.208310    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:05:31.208369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.208262    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:05:31.208369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.208340    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:05:31.208369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.208360    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:05:31.209607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.209594    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.209647 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.209618    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 22:05:31.210241 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.210226    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 22:05:31.210313 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.210250    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 22:05:31.210313 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.210260    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeHasSufficientPID"
Sep 10 22:05:31.237991 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.237971    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-25.ec2.internal\" not found" node="ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.242274 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.242260    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-25.ec2.internal\" not found" node="ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.254263 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.254249    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.272428 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.272407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c95898bd26ea01173288a950eebe286-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal\" (UID: \"3c95898bd26ea01173288a950eebe286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.272500 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.272431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c95898bd26ea01173288a950eebe286-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal\" (UID: \"3c95898bd26ea01173288a950eebe286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.272500 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.272463    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9996d37233f3f53259991758723d858-config\") pod \"kube-apiserver-proxy-ip-10-0-137-25.ec2.internal\" (UID: \"f9996d37233f3f53259991758723d858\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.354949 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.354908    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.373283 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.373254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c95898bd26ea01173288a950eebe286-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal\" (UID: \"3c95898bd26ea01173288a950eebe286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.373355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.373290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c95898bd26ea01173288a950eebe286-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal\" (UID: \"3c95898bd26ea01173288a950eebe286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.373355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.373306    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9996d37233f3f53259991758723d858-config\") pod \"kube-apiserver-proxy-ip-10-0-137-25.ec2.internal\" (UID: \"f9996d37233f3f53259991758723d858\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.373355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.373337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c95898bd26ea01173288a950eebe286-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal\" (UID: \"3c95898bd26ea01173288a950eebe286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.373355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.373344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3c95898bd26ea01173288a950eebe286-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal\" (UID: \"3c95898bd26ea01173288a950eebe286\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.373519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.373377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9996d37233f3f53259991758723d858-config\") pod \"kube-apiserver-proxy-ip-10-0-137-25.ec2.internal\" (UID: \"f9996d37233f3f53259991758723d858\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.455735 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.455664    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.540090 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.540061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.545483 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.545462    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.556400 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.556380    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.656916 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.656876    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.757523 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.757453    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-25.ec2.internal\" not found"
Sep 10 22:05:31.839251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.839228    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:05:31.870951 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.870913    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.874176 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.874154    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:05:31.886131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.886111    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 22:05:31.886948 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.886935    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal"
Sep 10 22:05:31.902782 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.902759    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 22:05:31.906904 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.906886    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 22:05:31.907043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.907020    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 22:05:31.907043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.907021    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 22:05:31.907161 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.907035    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 22:05:31.946213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.946185    2563 apiserver.go:52] "Watching apiserver"
Sep 10 22:05:31.970625 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.970602    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 22:05:31.987706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.987676    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 22:00:31 +0000 UTC" deadline="2028-06-09 00:34:58.47175375 +0000 UTC"
Sep 10 22:05:31.987706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.987702    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15290h29m26.484054679s"
Sep 10 22:05:31.988606 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.988587    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 22:05:31.988948 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.988930    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-4tpcc","openshift-network-operator/iptables-alerter-z62vc","openshift-ovn-kubernetes/ovnkube-node-6h452","kube-system/konnectivity-agent-dz966","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl","openshift-cluster-node-tuning-operator/tuned-fb7nj","openshift-dns/node-resolver-77qcj","openshift-multus/multus-additional-cni-plugins-4v5g8","openshift-multus/network-metrics-daemon-5gzpc","kube-system/global-pull-secret-syncer-k8cm5","kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal","openshift-image-registry/node-ca-vk5zq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal","openshift-multus/multus-lh9rq"]
Sep 10 22:05:31.990208 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.990192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:31.990281 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.990263    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:31.991343 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.991328    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:31.992423 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.992408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:31.993596 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.993578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:31.994767 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.994753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:31.995832 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.995816    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:31.996394 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.996377    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 22:05:31.996609 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.996595    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 22:05:31.996790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.996778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:31.997927 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.997914    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:31.998614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.998596    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 22:05:31.998614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.998610    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qsmdd\""
Sep 10 22:05:31.998737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.998626    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 22:05:31.999002 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:31.998987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:31.999085 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:31.999066    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:32.000308 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.000292    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.000389 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.000367    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:32.000704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.000686    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 22:05:32.000704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.000698    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 22:05:32.000822 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.000705    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 22:05:32.000822 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.000694    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 22:05:32.001402 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.001390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.001960 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.001939    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 22:05:32.002143 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.002126    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2cbt4\""
Sep 10 22:05:32.002262 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.002245    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 22:05:32.004406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.003008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.004406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.004098    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pbdpp\""
Sep 10 22:05:32.004406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.004361    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 22:05:32.004613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.004424    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 22:05:32.005567 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.005549    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2zdlw\""
Sep 10 22:05:32.005673 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.005586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 22:05:32.005673 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.005634    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 22:05:32.005785 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.005714    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 22:05:32.005855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.005838    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 22:05:32.007392 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.007365    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 22:05:32.007581 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.007399    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 22:05:32.007701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.007686    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-89pn5\""
Sep 10 22:05:32.007701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.007695    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 22:05:32.009665 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.009646    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 22:05:32.009665 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.009655    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 22:05:32.009779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.009682    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cjlzl\""
Sep 10 22:05:32.009779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.009682    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 22:05:32.010256 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.010244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 22:05:32.010356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.010341    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 22:05:32.010423 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.010343    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nplk2\""
Sep 10 22:05:32.010423 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.010377    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 22:05:32.010423 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.010385    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 22:05:32.010423 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.010388    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 22:05:32.012072 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.012058    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 22:05:32.012984 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.012968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gtppr\""
Sep 10 22:05:32.013053 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.013008    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sfgwp\""
Sep 10 22:05:32.025450 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.025414    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rfbwh"
Sep 10 22:05:32.032042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.031907    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rfbwh"
Sep 10 22:05:32.060523 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.060489    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c95898bd26ea01173288a950eebe286.slice/crio-527e291bfa9aebc54c51e491c8c44f638a9dfb0e5b99c0a1d373e5de337b474f WatchSource:0}: Error finding container 527e291bfa9aebc54c51e491c8c44f638a9dfb0e5b99c0a1d373e5de337b474f: Status 404 returned error can't find the container with id 527e291bfa9aebc54c51e491c8c44f638a9dfb0e5b99c0a1d373e5de337b474f
Sep 10 22:05:32.062263 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.062237    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9996d37233f3f53259991758723d858.slice/crio-d3adc4f5cad68a051aff8634544ec05c108bd952200b2649b7596aac54999966 WatchSource:0}: Error finding container d3adc4f5cad68a051aff8634544ec05c108bd952200b2649b7596aac54999966: Status 404 returned error can't find the container with id d3adc4f5cad68a051aff8634544ec05c108bd952200b2649b7596aac54999966
Sep 10 22:05:32.066874 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.066858    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 22:05:32.072049 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.072032    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 22:05:32.076784 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-env-overrides\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.076888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovn-node-metrics-cert\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.076888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-run\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.076888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc5rg\" (UniqueName: \"kubernetes.io/projected/0055c7c5-046b-47db-97e6-fb1b4349c8af-kube-api-access-kc5rg\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:32.076888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99a9cc07-00c0-4097-9166-5232c8ac3009-iptables-alerter-script\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-var-lib-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ddc1a73-dcad-4177-ae60-ed096db3624d-serviceca\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-kubelet\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.076994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-conf-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-kubelet\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-ovn\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovnkube-config\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f945n\" (UniqueName: \"kubernetes.io/projected/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-kube-api-access-f945n\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-lib-modules\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-var-lib-kubelet\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-daemon-config\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-run-netns\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-etc-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077280    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovnkube-script-lib\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b115e6d-22b2-422a-8e9a-2ab56b60e746-konnectivity-ca\") pod \"konnectivity-agent-dz966\" (UID: \"8b115e6d-22b2-422a-8e9a-2ab56b60e746\") " pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysctl-conf\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-tuned\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077392    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c58b53b6-ef61-400e-b3b4-beaf17da2443-hosts-file\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58b53b6-ef61-400e-b3b4-beaf17da2443-tmp-dir\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.077502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-system-cni-dir\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-cni-bin\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-cni-multus\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-log-socket\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-cni-bin\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p78ll\" (UniqueName: \"kubernetes.io/projected/99a9cc07-00c0-4097-9166-5232c8ac3009-kube-api-access-p78ll\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-socket-dir-parent\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077701    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077732    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysctl-d\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077785    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c2be52-0344-4199-90e5-0893092ad1ba-tmp\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e33414-abf8-4ebc-a2d2-22d60d529098-kubelet-config\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-systemd-units\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077852    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysconfig\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-sys\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.078123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xf56\" (UniqueName: \"kubernetes.io/projected/cb4095b3-21dd-4367-af3e-0aced8f43716-kube-api-access-9xf56\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6sns\" (UniqueName: \"kubernetes.io/projected/6e77c030-1627-4292-9879-37c85b306cd8-kube-api-access-g6sns\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.077990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhkzc\" (UniqueName: \"kubernetes.io/projected/a3c2be52-0344-4199-90e5-0893092ad1ba-kube-api-access-hhkzc\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-cnibin\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4bfg\" (UniqueName: \"kubernetes.io/projected/9ddc1a73-dcad-4177-ae60-ed096db3624d-kube-api-access-v4bfg\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-cni-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-multus-certs\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdkkd\" (UniqueName: \"kubernetes.io/projected/fec10a06-94be-41d8-b69e-0cf86b9958c5-kube-api-access-kdkkd\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078182    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-modprobe-d\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078233    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-kubernetes\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-cni-binary-copy\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078290    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-sys-fs\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99a9cc07-00c0-4097-9166-5232c8ac3009-host-slash\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-system-cni-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fec10a06-94be-41d8-b69e-0cf86b9958c5-cni-binary-copy\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.078709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-cni-netd\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e33414-abf8-4ebc-a2d2-22d60d529098-dbus\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ddc1a73-dcad-4177-ae60-ed096db3624d-host\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-cnibin\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-etc-kubernetes\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-node-log\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-device-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-systemd\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-k8s-cni-cncf-io\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078746    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-systemd\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078775    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-registration-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-host\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078834    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-socket-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6bf5\" (UniqueName: \"kubernetes.io/projected/c58b53b6-ef61-400e-b3b4-beaf17da2443-kube-api-access-p6bf5\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.079420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.080142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-slash\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.080142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.078986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b115e6d-22b2-422a-8e9a-2ab56b60e746-agent-certs\") pod \"konnectivity-agent-dz966\" (UID: \"8b115e6d-22b2-422a-8e9a-2ab56b60e746\") " pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:32.080142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.079007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:32.080142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.079030    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-netns\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.080142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.079058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-hostroot\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.107722 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.107689    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal" event={"ID":"3c95898bd26ea01173288a950eebe286","Type":"ContainerStarted","Data":"527e291bfa9aebc54c51e491c8c44f638a9dfb0e5b99c0a1d373e5de337b474f"}
Sep 10 22:05:32.108677 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.108658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal" event={"ID":"f9996d37233f3f53259991758723d858","Type":"ContainerStarted","Data":"d3adc4f5cad68a051aff8634544ec05c108bd952200b2649b7596aac54999966"}
Sep 10 22:05:32.180220 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-etc-kubernetes\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-node-log\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-device-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-systemd\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-k8s-cni-cncf-io\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180309    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-etc-kubernetes\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-systemd\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-node-log\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-registration-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180341    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-device-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180361    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-host\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-k8s-cni-cncf-io\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.180376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-systemd\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-systemd\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-socket-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180414    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-host\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-registration-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180468    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6bf5\" (UniqueName: \"kubernetes.io/projected/c58b53b6-ef61-400e-b3b4-beaf17da2443-kube-api-access-p6bf5\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180469    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-slash\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180561    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-slash\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-socket-dir\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b115e6d-22b2-422a-8e9a-2ab56b60e746-agent-certs\") pod \"konnectivity-agent-dz966\" (UID: \"8b115e6d-22b2-422a-8e9a-2ab56b60e746\") " pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-netns\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-hostroot\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.180928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-env-overrides\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovn-node-metrics-cert\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.180712    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-run\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc5rg\" (UniqueName: \"kubernetes.io/projected/0055c7c5-046b-47db-97e6-fb1b4349c8af-kube-api-access-kc5rg\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-netns\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.180783    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs podName:0055c7c5-046b-47db-97e6-fb1b4349c8af nodeName:}" failed. No retries permitted until 2026-09-10 22:05:32.680762724 +0000 UTC m=+2.065268129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs") pod "network-metrics-daemon-5gzpc" (UID: "0055c7c5-046b-47db-97e6-fb1b4349c8af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99a9cc07-00c0-4097-9166-5232c8ac3009-iptables-alerter-script\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180847    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-var-lib-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ddc1a73-dcad-4177-ae60-ed096db3624d-serviceca\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-kubelet\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-conf-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-kubelet\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-ovn\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovnkube-config\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.181692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f945n\" (UniqueName: \"kubernetes.io/projected/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-kube-api-access-f945n\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-run\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-lib-modules\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-var-lib-kubelet\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-daemon-config\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-env-overrides\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181131    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-run-netns\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-run-netns\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181147    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-var-lib-kubelet\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-etc-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181186    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-lib-modules\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-kubelet\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovnkube-script-lib\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-conf-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181235    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b115e6d-22b2-422a-8e9a-2ab56b60e746-konnectivity-ca\") pod \"konnectivity-agent-dz966\" (UID: \"8b115e6d-22b2-422a-8e9a-2ab56b60e746\") " pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.181259    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-ovn\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.182518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181258    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.181305    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret podName:f5e33414-abf8-4ebc-a2d2-22d60d529098 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:32.681290202 +0000 UTC m=+2.065795807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret") pod "global-pull-secret-syncer-k8cm5" (UID: "f5e33414-abf8-4ebc-a2d2-22d60d529098") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.180826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-hostroot\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181322    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-etc-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-var-lib-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181362    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysctl-conf\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-run-openvswitch\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysctl-conf\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-tuned\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c58b53b6-ef61-400e-b3b4-beaf17da2443-hosts-file\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58b53b6-ef61-400e-b3b4-beaf17da2443-tmp-dir\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-system-cni-dir\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181582    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-cni-bin\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.183466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-cni-multus\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-log-socket\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-cni-bin\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-daemon-config\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p78ll\" (UniqueName: \"kubernetes.io/projected/99a9cc07-00c0-4097-9166-5232c8ac3009-kube-api-access-p78ll\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-socket-dir-parent\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181707    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-kubelet\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181755    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysctl-d\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c2be52-0344-4199-90e5-0893092ad1ba-tmp\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181802    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e33414-abf8-4ebc-a2d2-22d60d529098-kubelet-config\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-systemd-units\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysconfig\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-sys\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181923    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b115e6d-22b2-422a-8e9a-2ab56b60e746-konnectivity-ca\") pod \"konnectivity-agent-dz966\" (UID: \"8b115e6d-22b2-422a-8e9a-2ab56b60e746\") " pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xf56\" (UniqueName: \"kubernetes.io/projected/cb4095b3-21dd-4367-af3e-0aced8f43716-kube-api-access-9xf56\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.184226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6sns\" (UniqueName: \"kubernetes.io/projected/6e77c030-1627-4292-9879-37c85b306cd8-kube-api-access-g6sns\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhkzc\" (UniqueName: \"kubernetes.io/projected/a3c2be52-0344-4199-90e5-0893092ad1ba-kube-api-access-hhkzc\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-cnibin\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4bfg\" (UniqueName: \"kubernetes.io/projected/9ddc1a73-dcad-4177-ae60-ed096db3624d-kube-api-access-v4bfg\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-cni-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-multus-certs\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysctl-d\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdkkd\" (UniqueName: \"kubernetes.io/projected/fec10a06-94be-41d8-b69e-0cf86b9958c5-kube-api-access-kdkkd\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182135    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c58b53b6-ef61-400e-b3b4-beaf17da2443-hosts-file\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-cni-bin\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-modprobe-d\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-kubernetes\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-var-lib-cni-multus\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-cni-binary-copy\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-log-socket\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.185007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182252    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-sys-fs\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182260    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovnkube-script-lib\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182290    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-cni-bin\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182343    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-socket-dir-parent\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c58b53b6-ef61-400e-b3b4-beaf17da2443-tmp-dir\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182480    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-modprobe-d\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-system-cni-dir\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182557    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-kubernetes\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-multus-cni-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6e77c030-1627-4292-9879-37c85b306cd8-sys-fs\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182647    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99a9cc07-00c0-4097-9166-5232c8ac3009-host-slash\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-system-cni-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fec10a06-94be-41d8-b69e-0cf86b9958c5-cni-binary-copy\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e33414-abf8-4ebc-a2d2-22d60d529098-kubelet-config\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-cni-netd\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182733    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-host-run-multus-certs\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.185613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9ddc1a73-dcad-4177-ae60-ed096db3624d-serviceca\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e33414-abf8-4ebc-a2d2-22d60d529098-dbus\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovnkube-config\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-systemd-units\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182647    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4095b3-21dd-4367-af3e-0aced8f43716-cnibin\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.181833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/99a9cc07-00c0-4097-9166-5232c8ac3009-iptables-alerter-script\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/99a9cc07-00c0-4097-9166-5232c8ac3009-host-slash\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-sysconfig\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182887    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-host-cni-netd\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182933    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-system-cni-dir\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3c2be52-0344-4199-90e5-0893092ad1ba-sys\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.182982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ddc1a73-dcad-4177-ae60-ed096db3624d-host\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.183002    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4095b3-21dd-4367-af3e-0aced8f43716-cni-binary-copy\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.183008    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e33414-abf8-4ebc-a2d2-22d60d529098-dbus\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.183007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-cnibin\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.183034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9ddc1a73-dcad-4177-ae60-ed096db3624d-host\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.183048    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fec10a06-94be-41d8-b69e-0cf86b9958c5-cnibin\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.186150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.183390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fec10a06-94be-41d8-b69e-0cf86b9958c5-cni-binary-copy\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.186666 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.185299    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3c2be52-0344-4199-90e5-0893092ad1ba-etc-tuned\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.186666 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.185427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3c2be52-0344-4199-90e5-0893092ad1ba-tmp\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.186666 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.185525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-ovn-node-metrics-cert\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.186666 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.185633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b115e6d-22b2-422a-8e9a-2ab56b60e746-agent-certs\") pod \"konnectivity-agent-dz966\" (UID: \"8b115e6d-22b2-422a-8e9a-2ab56b60e746\") " pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:32.201190 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.201167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6bf5\" (UniqueName: \"kubernetes.io/projected/c58b53b6-ef61-400e-b3b4-beaf17da2443-kube-api-access-p6bf5\") pod \"node-resolver-77qcj\" (UID: \"c58b53b6-ef61-400e-b3b4-beaf17da2443\") " pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.201318 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.201300    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:05:32.201376 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.201324    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:05:32.201376 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.201339    2563 projected.go:196] Error preparing data for projected volume kube-api-access-878c9 for pod openshift-network-diagnostics/network-check-target-4tpcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:32.201581 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.201417    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9 podName:0a8500ef-406f-4329-acd2-0e6d202399a5 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:32.70139806 +0000 UTC m=+2.085903463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-878c9" (UniqueName: "kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9") pod "network-check-target-4tpcc" (UID: "0a8500ef-406f-4329-acd2-0e6d202399a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:32.203652 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.203545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdkkd\" (UniqueName: \"kubernetes.io/projected/fec10a06-94be-41d8-b69e-0cf86b9958c5-kube-api-access-kdkkd\") pod \"multus-lh9rq\" (UID: \"fec10a06-94be-41d8-b69e-0cf86b9958c5\") " pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.204207 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.203999    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p78ll\" (UniqueName: \"kubernetes.io/projected/99a9cc07-00c0-4097-9166-5232c8ac3009-kube-api-access-p78ll\") pod \"iptables-alerter-z62vc\" (UID: \"99a9cc07-00c0-4097-9166-5232c8ac3009\") " pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.204207 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.204026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4bfg\" (UniqueName: \"kubernetes.io/projected/9ddc1a73-dcad-4177-ae60-ed096db3624d-kube-api-access-v4bfg\") pod \"node-ca-vk5zq\" (UID: \"9ddc1a73-dcad-4177-ae60-ed096db3624d\") " pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.204425 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.204408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xf56\" (UniqueName: \"kubernetes.io/projected/cb4095b3-21dd-4367-af3e-0aced8f43716-kube-api-access-9xf56\") pod \"multus-additional-cni-plugins-4v5g8\" (UID: \"cb4095b3-21dd-4367-af3e-0aced8f43716\") " pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.204502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.204470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f945n\" (UniqueName: \"kubernetes.io/projected/f5d0a57a-deb3-4fe5-9121-71b7526e7fb3-kube-api-access-f945n\") pod \"ovnkube-node-6h452\" (UID: \"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.204898 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.204880    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6sns\" (UniqueName: \"kubernetes.io/projected/6e77c030-1627-4292-9879-37c85b306cd8-kube-api-access-g6sns\") pod \"aws-ebs-csi-driver-node-5pdjl\" (UID: \"6e77c030-1627-4292-9879-37c85b306cd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.208140 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.208122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc5rg\" (UniqueName: \"kubernetes.io/projected/0055c7c5-046b-47db-97e6-fb1b4349c8af-kube-api-access-kc5rg\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:32.210263 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.210247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhkzc\" (UniqueName: \"kubernetes.io/projected/a3c2be52-0344-4199-90e5-0893092ad1ba-kube-api-access-hhkzc\") pod \"tuned-fb7nj\" (UID: \"a3c2be52-0344-4199-90e5-0893092ad1ba\") " pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.315662 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.315590    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:05:32.317579 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.317562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z62vc"
Sep 10 22:05:32.322119 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.322101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:32.327645 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.327625    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a9cc07_00c0_4097_9166_5232c8ac3009.slice/crio-340b8821187f3ba0c44e0e5a6fededc81f028e12b7cc783347abe91c04e658ec WatchSource:0}: Error finding container 340b8821187f3ba0c44e0e5a6fededc81f028e12b7cc783347abe91c04e658ec: Status 404 returned error can't find the container with id 340b8821187f3ba0c44e0e5a6fededc81f028e12b7cc783347abe91c04e658ec
Sep 10 22:05:32.331392 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.331371    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d0a57a_deb3_4fe5_9121_71b7526e7fb3.slice/crio-e85d09703cf8e863064a2aaa1778bb869729012ce6974ac0dab4104d438652ac WatchSource:0}: Error finding container e85d09703cf8e863064a2aaa1778bb869729012ce6974ac0dab4104d438652ac: Status 404 returned error can't find the container with id e85d09703cf8e863064a2aaa1778bb869729012ce6974ac0dab4104d438652ac
Sep 10 22:05:32.344731 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.344711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:32.353207 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.353059    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b115e6d_22b2_422a_8e9a_2ab56b60e746.slice/crio-ffe14dcdea7543f9c685aa8ede7883a8df7ee15b2fc6165d6652beec40896fef WatchSource:0}: Error finding container ffe14dcdea7543f9c685aa8ede7883a8df7ee15b2fc6165d6652beec40896fef: Status 404 returned error can't find the container with id ffe14dcdea7543f9c685aa8ede7883a8df7ee15b2fc6165d6652beec40896fef
Sep 10 22:05:32.368661 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.368644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl"
Sep 10 22:05:32.375531 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.375513    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fb7nj"
Sep 10 22:05:32.376993 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.376965    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e77c030_1627_4292_9879_37c85b306cd8.slice/crio-8d176e1ed9bae02dd380dd255bb92dc9e70773440aed52ac94f94d33d3fa828e WatchSource:0}: Error finding container 8d176e1ed9bae02dd380dd255bb92dc9e70773440aed52ac94f94d33d3fa828e: Status 404 returned error can't find the container with id 8d176e1ed9bae02dd380dd255bb92dc9e70773440aed52ac94f94d33d3fa828e
Sep 10 22:05:32.381359 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.381340    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-77qcj"
Sep 10 22:05:32.384383 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.384347    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3c2be52_0344_4199_90e5_0893092ad1ba.slice/crio-1ca67f19311eb6f29dfa013a09314802b91f3ed9a2725fb39b6315c39369f67b WatchSource:0}: Error finding container 1ca67f19311eb6f29dfa013a09314802b91f3ed9a2725fb39b6315c39369f67b: Status 404 returned error can't find the container with id 1ca67f19311eb6f29dfa013a09314802b91f3ed9a2725fb39b6315c39369f67b
Sep 10 22:05:32.386887 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.386870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4v5g8"
Sep 10 22:05:32.392149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.392127    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vk5zq"
Sep 10 22:05:32.392670 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.392648    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58b53b6_ef61_400e_b3b4_beaf17da2443.slice/crio-c8f5197cbefdf30e632e17fe062a766015988834b7399e97abce77b562ffdb08 WatchSource:0}: Error finding container c8f5197cbefdf30e632e17fe062a766015988834b7399e97abce77b562ffdb08: Status 404 returned error can't find the container with id c8f5197cbefdf30e632e17fe062a766015988834b7399e97abce77b562ffdb08
Sep 10 22:05:32.399196 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.396781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lh9rq"
Sep 10 22:05:32.400204 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.400176    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4095b3_21dd_4367_af3e_0aced8f43716.slice/crio-458e105585946425e863ca0b7e7d0ff4b9fbc3fb8a04e055aff4cc0d48869ef1 WatchSource:0}: Error finding container 458e105585946425e863ca0b7e7d0ff4b9fbc3fb8a04e055aff4cc0d48869ef1: Status 404 returned error can't find the container with id 458e105585946425e863ca0b7e7d0ff4b9fbc3fb8a04e055aff4cc0d48869ef1
Sep 10 22:05:32.403572 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.403550    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ddc1a73_dcad_4177_ae60_ed096db3624d.slice/crio-f99e91c1d7bff5a70dfaa0a103a867a70a5bf253f4b8d597c6a6d4031c104f92 WatchSource:0}: Error finding container f99e91c1d7bff5a70dfaa0a103a867a70a5bf253f4b8d597c6a6d4031c104f92: Status 404 returned error can't find the container with id f99e91c1d7bff5a70dfaa0a103a867a70a5bf253f4b8d597c6a6d4031c104f92
Sep 10 22:05:32.408659 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:05:32.408628    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec10a06_94be_41d8_b69e_0cf86b9958c5.slice/crio-da189f70a6abdee43cce3e94cc799082ee7f4c751dd1c44732eed8107004e697 WatchSource:0}: Error finding container da189f70a6abdee43cce3e94cc799082ee7f4c751dd1c44732eed8107004e697: Status 404 returned error can't find the container with id da189f70a6abdee43cce3e94cc799082ee7f4c751dd1c44732eed8107004e697
Sep 10 22:05:32.686656 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.686541    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:32.686815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.686692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:32.686815 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.686729    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:32.686815 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.686798    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret podName:f5e33414-abf8-4ebc-a2d2-22d60d529098 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:33.686781383 +0000 UTC m=+3.071286811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret") pod "global-pull-secret-syncer-k8cm5" (UID: "f5e33414-abf8-4ebc-a2d2-22d60d529098") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:32.686815 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.686804    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:32.687049 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.686867    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs podName:0055c7c5-046b-47db-97e6-fb1b4349c8af nodeName:}" failed. No retries permitted until 2026-09-10 22:05:33.686849114 +0000 UTC m=+3.071354525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs") pod "network-metrics-daemon-5gzpc" (UID: "0055c7c5-046b-47db-97e6-fb1b4349c8af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:32.788532 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:32.787767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:32.788532 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.788012    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:05:32.788532 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.788033    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:05:32.788532 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.788046    2563 projected.go:196] Error preparing data for projected volume kube-api-access-878c9 for pod openshift-network-diagnostics/network-check-target-4tpcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:32.788532 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:32.788103    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9 podName:0a8500ef-406f-4329-acd2-0e6d202399a5 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:33.788084252 +0000 UTC m=+3.172589662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-878c9" (UniqueName: "kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9") pod "network-check-target-4tpcc" (UID: "0a8500ef-406f-4329-acd2-0e6d202399a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:33.033498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.033378    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 22:00:33 +0000 UTC" deadline="2028-03-05 22:53:07.814479677 +0000 UTC"
Sep 10 22:05:33.033498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.033460    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13008h47m34.781040572s"
Sep 10 22:05:33.117801 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.117739    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:05:33.129458 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.129414    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z62vc" event={"ID":"99a9cc07-00c0-4097-9166-5232c8ac3009","Type":"ContainerStarted","Data":"340b8821187f3ba0c44e0e5a6fededc81f028e12b7cc783347abe91c04e658ec"}
Sep 10 22:05:33.133569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.133531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vk5zq" event={"ID":"9ddc1a73-dcad-4177-ae60-ed096db3624d","Type":"ContainerStarted","Data":"f99e91c1d7bff5a70dfaa0a103a867a70a5bf253f4b8d597c6a6d4031c104f92"}
Sep 10 22:05:33.143924 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.143886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"e85d09703cf8e863064a2aaa1778bb869729012ce6974ac0dab4104d438652ac"}
Sep 10 22:05:33.166053 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.166013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lh9rq" event={"ID":"fec10a06-94be-41d8-b69e-0cf86b9958c5","Type":"ContainerStarted","Data":"da189f70a6abdee43cce3e94cc799082ee7f4c751dd1c44732eed8107004e697"}
Sep 10 22:05:33.182753 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.182655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerStarted","Data":"458e105585946425e863ca0b7e7d0ff4b9fbc3fb8a04e055aff4cc0d48869ef1"}
Sep 10 22:05:33.206955 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.206885    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-77qcj" event={"ID":"c58b53b6-ef61-400e-b3b4-beaf17da2443","Type":"ContainerStarted","Data":"c8f5197cbefdf30e632e17fe062a766015988834b7399e97abce77b562ffdb08"}
Sep 10 22:05:33.214681 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.214649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fb7nj" event={"ID":"a3c2be52-0344-4199-90e5-0893092ad1ba","Type":"ContainerStarted","Data":"1ca67f19311eb6f29dfa013a09314802b91f3ed9a2725fb39b6315c39369f67b"}
Sep 10 22:05:33.222786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.222202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl" event={"ID":"6e77c030-1627-4292-9879-37c85b306cd8","Type":"ContainerStarted","Data":"8d176e1ed9bae02dd380dd255bb92dc9e70773440aed52ac94f94d33d3fa828e"}
Sep 10 22:05:33.236201 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.236171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dz966" event={"ID":"8b115e6d-22b2-422a-8e9a-2ab56b60e746","Type":"ContainerStarted","Data":"ffe14dcdea7543f9c685aa8ede7883a8df7ee15b2fc6165d6652beec40896fef"}
Sep 10 22:05:33.695273 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.695237    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:33.695499 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.695290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:33.695499 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.695420    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:33.695647 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.695501    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret podName:f5e33414-abf8-4ebc-a2d2-22d60d529098 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:35.695462812 +0000 UTC m=+5.079968237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret") pod "global-pull-secret-syncer-k8cm5" (UID: "f5e33414-abf8-4ebc-a2d2-22d60d529098") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:33.695927 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.695906    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:33.696013 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.695967    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs podName:0055c7c5-046b-47db-97e6-fb1b4349c8af nodeName:}" failed. No retries permitted until 2026-09-10 22:05:35.695950203 +0000 UTC m=+5.080455612 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs") pod "network-metrics-daemon-5gzpc" (UID: "0055c7c5-046b-47db-97e6-fb1b4349c8af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:33.795941 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:33.795906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:33.796111 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.796079    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:05:33.796111 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.796105    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:05:33.796244 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.796119    2563 projected.go:196] Error preparing data for projected volume kube-api-access-878c9 for pod openshift-network-diagnostics/network-check-target-4tpcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:33.796244 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:33.796198    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9 podName:0a8500ef-406f-4329-acd2-0e6d202399a5 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:35.796160822 +0000 UTC m=+5.180666252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-878c9" (UniqueName: "kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9") pod "network-check-target-4tpcc" (UID: "0a8500ef-406f-4329-acd2-0e6d202399a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:34.034125 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:34.034023    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 22:00:33 +0000 UTC" deadline="2028-02-28 21:25:52.581902579 +0000 UTC"
Sep 10 22:05:34.034125 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:34.034073    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12863h20m18.547834266s"
Sep 10 22:05:34.104355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:34.104320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:34.104619 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:34.104492    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:34.104988 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:34.104909    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:34.105067 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:34.105015    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:34.105108 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:34.105078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:34.105163 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:34.105144    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:34.813310 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:34.813277    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:05:34.914769 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:34.914735    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 22:05:35.713420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:35.713383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:35.714050 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:35.713432    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:35.714050 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.713557    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:35.714050 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.713586    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:35.714050 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.713621    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs podName:0055c7c5-046b-47db-97e6-fb1b4349c8af nodeName:}" failed. No retries permitted until 2026-09-10 22:05:39.713601403 +0000 UTC m=+9.098106822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs") pod "network-metrics-daemon-5gzpc" (UID: "0055c7c5-046b-47db-97e6-fb1b4349c8af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:35.714050 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.713641    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret podName:f5e33414-abf8-4ebc-a2d2-22d60d529098 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:39.713630647 +0000 UTC m=+9.098136053 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret") pod "global-pull-secret-syncer-k8cm5" (UID: "f5e33414-abf8-4ebc-a2d2-22d60d529098") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:35.814918 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:35.814885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:35.815107 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.815070    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:05:35.815107 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.815094    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:05:35.815107 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.815106    2563 projected.go:196] Error preparing data for projected volume kube-api-access-878c9 for pod openshift-network-diagnostics/network-check-target-4tpcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:35.815275 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:35.815179    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9 podName:0a8500ef-406f-4329-acd2-0e6d202399a5 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:39.815147229 +0000 UTC m=+9.199652649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-878c9" (UniqueName: "kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9") pod "network-check-target-4tpcc" (UID: "0a8500ef-406f-4329-acd2-0e6d202399a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:36.104900 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:36.104807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:36.105070 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:36.104972    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:36.105474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:36.105447    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:36.105624 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:36.105576    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:36.105697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:36.105639    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:36.105804 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:36.105762    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:38.104655 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:38.104620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:38.105122 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:38.104620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:38.105122 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:38.104764    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:38.105122 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:38.104634    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:38.105122 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:38.104813    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:38.105122 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:38.104931    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:39.747855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:39.747816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:39.748319 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:39.747869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:39.748319 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.747991    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:39.748319 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.748057    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret podName:f5e33414-abf8-4ebc-a2d2-22d60d529098 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:47.748036351 +0000 UTC m=+17.132541762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret") pod "global-pull-secret-syncer-k8cm5" (UID: "f5e33414-abf8-4ebc-a2d2-22d60d529098") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:39.748541 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.748429    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:39.748541 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.748466    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs podName:0055c7c5-046b-47db-97e6-fb1b4349c8af nodeName:}" failed. No retries permitted until 2026-09-10 22:05:47.748454307 +0000 UTC m=+17.132959710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs") pod "network-metrics-daemon-5gzpc" (UID: "0055c7c5-046b-47db-97e6-fb1b4349c8af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:39.848790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:39.848755    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:39.848988 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.848966    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:05:39.849073 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.848997    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:05:39.849073 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.849012    2563 projected.go:196] Error preparing data for projected volume kube-api-access-878c9 for pod openshift-network-diagnostics/network-check-target-4tpcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:39.849184 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:39.849081    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9 podName:0a8500ef-406f-4329-acd2-0e6d202399a5 nodeName:}" failed. No retries permitted until 2026-09-10 22:05:47.849060898 +0000 UTC m=+17.233566321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-878c9" (UniqueName: "kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9") pod "network-check-target-4tpcc" (UID: "0a8500ef-406f-4329-acd2-0e6d202399a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:40.104734 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:40.104640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:40.104734 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:40.104662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:40.104975 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:40.104651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:40.104975 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:40.104792    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:40.104975 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:40.104889    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:40.105136 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:40.104993    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:42.104867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:42.104826    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:42.104867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:42.104863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:42.105379 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:42.104822    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:42.105379 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:42.104971    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:42.105379 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:42.105073    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:42.105379 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:42.105142    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:44.104454 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:44.104392    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:44.104950 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:44.104463    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:44.104950 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:44.104526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:44.104950 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:44.104543    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:44.104950 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:44.104654    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:44.104950 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:44.104745    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:46.106471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:46.106425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:46.106943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:46.106426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:46.106943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:46.106429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:46.106943 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:46.106769    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:46.106943 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:46.106828    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:46.106943 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:46.106899    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:47.816799 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:47.816760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:47.817409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:47.816815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:47.817409 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.816924    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:47.817409 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.816954    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:47.817409 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.816989    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs podName:0055c7c5-046b-47db-97e6-fb1b4349c8af nodeName:}" failed. No retries permitted until 2026-09-10 22:06:03.816971555 +0000 UTC m=+33.201476962 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs") pod "network-metrics-daemon-5gzpc" (UID: "0055c7c5-046b-47db-97e6-fb1b4349c8af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:05:47.817409 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.817011    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret podName:f5e33414-abf8-4ebc-a2d2-22d60d529098 nodeName:}" failed. No retries permitted until 2026-09-10 22:06:03.817000383 +0000 UTC m=+33.201505786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret") pod "global-pull-secret-syncer-k8cm5" (UID: "f5e33414-abf8-4ebc-a2d2-22d60d529098") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:05:47.917420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:47.917381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:47.917625 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.917586    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:05:47.917625 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.917610    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:05:47.917625 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.917623    2563 projected.go:196] Error preparing data for projected volume kube-api-access-878c9 for pod openshift-network-diagnostics/network-check-target-4tpcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:47.917782 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:47.917688    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9 podName:0a8500ef-406f-4329-acd2-0e6d202399a5 nodeName:}" failed. No retries permitted until 2026-09-10 22:06:03.917669273 +0000 UTC m=+33.302174677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-878c9" (UniqueName: "kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9") pod "network-check-target-4tpcc" (UID: "0a8500ef-406f-4329-acd2-0e6d202399a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:05:48.104547 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:48.104464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:48.104547 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:48.104525    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:48.104754 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:48.104551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:48.104754 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:48.104654    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:48.104961 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:48.104916    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:48.105078 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:48.105016    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:50.104594 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:50.104555    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:50.104594 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:50.104582    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:50.105069 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:50.104585    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:50.105069 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:50.104693    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:50.105069 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:50.104753    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:50.105069 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:50.104813    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:52.104988 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:52.104960    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:52.105241 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:52.105000    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:52.105241 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:52.105001    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:52.105241 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:52.105076    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:52.105241 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:52.105153    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:52.105241 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:52.105213    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:53.334395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.333990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vk5zq" event={"ID":"9ddc1a73-dcad-4177-ae60-ed096db3624d","Type":"ContainerStarted","Data":"67aa7785eb8493f3fad6813f7d86c64a6b31d8da52f6cd05a5f481a2207080d3"}
Sep 10 22:05:53.337102 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.337080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"30dbc4fe9509c3a0b776d5daeeb59e6ad8efc78893a3175641bed06dcd6b2493"}
Sep 10 22:05:53.337217 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.337145    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"beab3338b6fd22eddf211295920f63a50f5f91685d62284f73b59743fbc5a9e5"}
Sep 10 22:05:53.337217 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.337177    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"0805f3bd9a6e0990a2c71cddd918170b99a9fe9d572d6fbe68748f74e24f67e4"}
Sep 10 22:05:53.337308 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.337222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"112df8a131dda7863aa92a751dde355fc08d121f228fb0dc6192b2bcf5846043"}
Sep 10 22:05:53.337308 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.337264    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"0848ae4d71a162f4c7c26261b5cb0955eeedc80c38642143490313b3219e1d1c"}
Sep 10 22:05:53.337379 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.337306    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"e955c2c5b062817b093fc8b1e76083901c2e94be7130815f1f26eb13e44456a0"}
Sep 10 22:05:53.338511 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.338486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal" event={"ID":"f9996d37233f3f53259991758723d858","Type":"ContainerStarted","Data":"2143fd2c25d657f8e248ae941b3700a3a1de1a9ee2f42d3a88fc3ea90c41ecce"}
Sep 10 22:05:53.339950 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.339929    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lh9rq" event={"ID":"fec10a06-94be-41d8-b69e-0cf86b9958c5","Type":"ContainerStarted","Data":"07f76697b8d69b9f3c6c0b91fa50a1abe816a2e1d6c9c8d7c7e2b354e1799667"}
Sep 10 22:05:53.341401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.341379    2563 generic.go:356] "Generic (PLEG): container finished" podID="cb4095b3-21dd-4367-af3e-0aced8f43716" containerID="07c91de61e3f14486e914a126999b48fe6bb46463f4f7f0480580dbb35c0d99b" exitCode=0
Sep 10 22:05:53.341506 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.341463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerDied","Data":"07c91de61e3f14486e914a126999b48fe6bb46463f4f7f0480580dbb35c0d99b"}
Sep 10 22:05:53.343000 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.342983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-77qcj" event={"ID":"c58b53b6-ef61-400e-b3b4-beaf17da2443","Type":"ContainerStarted","Data":"03b09ab394c094aeb247c5b5fcc298ca6afe03aefc3deb766738f922a8d66da0"}
Sep 10 22:05:53.344641 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.344619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fb7nj" event={"ID":"a3c2be52-0344-4199-90e5-0893092ad1ba","Type":"ContainerStarted","Data":"b5fd17b6c245c8061b95704907dfb4575efc63a19fded16692a813427fcb9305"}
Sep 10 22:05:53.346007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.345982    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl" event={"ID":"6e77c030-1627-4292-9879-37c85b306cd8","Type":"ContainerStarted","Data":"b5b39c99dede78b370406cbdf5239b40190337a1317fd5b2e047f01bf790365d"}
Sep 10 22:05:53.347284 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.347266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dz966" event={"ID":"8b115e6d-22b2-422a-8e9a-2ab56b60e746","Type":"ContainerStarted","Data":"d5907773f1c6d267b106ce625bb65a28aec83aea05442d1bce987fdda938f78c"}
Sep 10 22:05:53.348755 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.348737    2563 generic.go:356] "Generic (PLEG): container finished" podID="3c95898bd26ea01173288a950eebe286" containerID="8bbb2b5356d18abaaf0a034a57b4bbe4235c4a3ebaeb1414b2ed162ce2003999" exitCode=0
Sep 10 22:05:53.348825 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.348762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal" event={"ID":"3c95898bd26ea01173288a950eebe286","Type":"ContainerDied","Data":"8bbb2b5356d18abaaf0a034a57b4bbe4235c4a3ebaeb1414b2ed162ce2003999"}
Sep 10 22:05:53.392553 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.392509    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vk5zq" podStartSLOduration=1.653416908 podStartE2EDuration="21.392494502s" totalImagesPullingTime="19.739077594s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:53.392014551 +0000 UTC m=+22.776519977" watchObservedRunningTime="2026-09-10 22:05:53.392494502 +0000 UTC m=+22.776999935"
Sep 10 22:05:53.438709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.438662    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-25.ec2.internal" podStartSLOduration=21.438642133 podStartE2EDuration="21.438642133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:53.438485706 +0000 UTC m=+22.822991128" watchObservedRunningTime="2026-09-10 22:05:53.438642133 +0000 UTC m=+22.823147560"
Sep 10 22:05:53.496648 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.496596    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fb7nj" podStartSLOduration=1.76683739 podStartE2EDuration="21.496574637s" totalImagesPullingTime="19.729737247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:53.495825567 +0000 UTC m=+22.880330999" watchObservedRunningTime="2026-09-10 22:05:53.496574637 +0000 UTC m=+22.881080064"
Sep 10 22:05:53.665014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.664948    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dz966" podStartSLOduration=1.87432188 podStartE2EDuration="21.664926866s" totalImagesPullingTime="19.790604986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:53.664619405 +0000 UTC m=+23.049124830" watchObservedRunningTime="2026-09-10 22:05:53.664926866 +0000 UTC m=+23.049432294"
Sep 10 22:05:53.792410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.792331    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lh9rq" podStartSLOduration=1.94252566 podStartE2EDuration="21.792315214s" totalImagesPullingTime="19.849789554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:53.79215144 +0000 UTC m=+23.176656865" watchObservedRunningTime="2026-09-10 22:05:53.792315214 +0000 UTC m=+23.176820639"
Sep 10 22:05:53.792762 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:53.792739    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-77qcj" podStartSLOduration=2.07292153 podStartE2EDuration="21.792733326s" totalImagesPullingTime="19.719811796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:53.713957229 +0000 UTC m=+23.098462655" watchObservedRunningTime="2026-09-10 22:05:53.792733326 +0000 UTC m=+23.177238751"
Sep 10 22:05:54.104872 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.104841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:54.105019 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.104884    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:54.105019 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.104841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:54.105019 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:54.104976    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:54.105169 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:54.105098    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:54.105226 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:54.105175    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:54.146293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.146268    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:54.188961 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.188941    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 22:05:54.353086 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.353001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl" event={"ID":"6e77c030-1627-4292-9879-37c85b306cd8","Type":"ContainerStarted","Data":"1794847dc000e5cf4356c715e1b70040e6ec8652b1c19619d588e5ef13282542"}
Sep 10 22:05:54.354776 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.354751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z62vc" event={"ID":"99a9cc07-00c0-4097-9166-5232c8ac3009","Type":"ContainerStarted","Data":"d70d6bbb6b61b0132ccc7d8361612b1e56f9fad18e646c46034679bb9c137d03"}
Sep 10 22:05:54.356756 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.356720    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal" event={"ID":"3c95898bd26ea01173288a950eebe286","Type":"ContainerStarted","Data":"705a6a87945d0f7e26cbc9318f0cc0260cfe59f50431f493446d4c44730e9b48"}
Sep 10 22:05:54.398482 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.398415    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-25.ec2.internal" podStartSLOduration=22.398397671 podStartE2EDuration="22.398397671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:54.398160622 +0000 UTC m=+23.782666048" watchObservedRunningTime="2026-09-10 22:05:54.398397671 +0000 UTC m=+23.782903097"
Sep 10 22:05:54.398654 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:54.398633    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z62vc" podStartSLOduration=2.580225905 podStartE2EDuration="22.39862698s" totalImagesPullingTime="19.818401075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:54.375838666 +0000 UTC m=+23.760344092" watchObservedRunningTime="2026-09-10 22:05:54.39862698 +0000 UTC m=+23.783132456"
Sep 10 22:05:55.043968 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:55.043717    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T22:05:54.188956315Z","UUID":"2721da3a-c13e-46ab-b438-b64dfbe990d2","Handler":null,"Name":"","Endpoint":""}
Sep 10 22:05:55.045284 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:55.045261    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 22:05:55.045595 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:55.045299    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 22:05:55.363687 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:55.363567    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"02fecb005c96070d988c325dd39ec7e416e7cbd7737e6236c777570d9d27f8ad"}
Sep 10 22:05:56.104503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.104463    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:56.104708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.104481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:56.104708 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:56.104593    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:56.104708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.104608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:56.104888 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:56.104713    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:56.104888 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:56.104799    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:56.190539 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.190509    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:56.191285 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.191263    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:56.369121 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.369047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl" event={"ID":"6e77c030-1627-4292-9879-37c85b306cd8","Type":"ContainerStarted","Data":"4aaf0b31bbbbdc65a44f15fd3479be8c57c2512141e70cbeb63846da1aaa9e46"}
Sep 10 22:05:56.369749 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.369727    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dz966"
Sep 10 22:05:56.407583 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:56.407529    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5pdjl" podStartSLOduration=2.050833573 podStartE2EDuration="24.40751151s" totalImagesPullingTime="22.356677937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:56.398994627 +0000 UTC m=+25.783500052" watchObservedRunningTime="2026-09-10 22:05:56.40751151 +0000 UTC m=+25.792016936"
Sep 10 22:05:58.104643 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.104625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:05:58.104993 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.104625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:05:58.104993 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:58.104734    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:05:58.104993 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.104625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:05:58.104993 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:58.104848    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:05:58.104993 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:05:58.104788    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:05:58.377832 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.377562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" event={"ID":"f5d0a57a-deb3-4fe5-9121-71b7526e7fb3","Type":"ContainerStarted","Data":"d937a6cc6a4cf64b452abba556771ce8f548be0e4f00c46359689ea075f68be4"}
Sep 10 22:05:58.377964 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.377856    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:58.377964 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.377907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:58.377964 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.377947    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:58.380146 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.380122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerStarted","Data":"0200fbe72c1dd77e9c457473d57e91723e7d918070c09bd9ea97130a8b7e73a2"}
Sep 10 22:05:58.394020 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.393995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:58.394165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.394151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:05:58.420852 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:58.420812    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6h452" podStartSLOduration=6.479737094 podStartE2EDuration="26.420799161s" totalImagesPullingTime="19.941062067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:05:58.420514027 +0000 UTC m=+27.805019451" watchObservedRunningTime="2026-09-10 22:05:58.420799161 +0000 UTC m=+27.805304586"
Sep 10 22:05:59.385076 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:59.385037    2563 generic.go:356] "Generic (PLEG): container finished" podID="cb4095b3-21dd-4367-af3e-0aced8f43716" containerID="0200fbe72c1dd77e9c457473d57e91723e7d918070c09bd9ea97130a8b7e73a2" exitCode=0
Sep 10 22:05:59.385600 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:05:59.385123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerDied","Data":"0200fbe72c1dd77e9c457473d57e91723e7d918070c09bd9ea97130a8b7e73a2"}
Sep 10 22:06:00.033616 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.033580    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5gzpc"]
Sep 10 22:06:00.033791 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.033713    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:00.033853 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:00.033812    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:06:00.037879 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.037856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k8cm5"]
Sep 10 22:06:00.038007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.037981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:00.038103 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:00.038084    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:06:00.038589 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.038569    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4tpcc"]
Sep 10 22:06:00.038699 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.038653    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:00.038781 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:00.038724    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:06:00.389121 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.389044    2563 generic.go:356] "Generic (PLEG): container finished" podID="cb4095b3-21dd-4367-af3e-0aced8f43716" containerID="c05e4a83e8c9b48bf05c671fd0b188277530f6787bcd8c4c1156d059206ad103" exitCode=0
Sep 10 22:06:00.389819 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:00.389120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerDied","Data":"c05e4a83e8c9b48bf05c671fd0b188277530f6787bcd8c4c1156d059206ad103"}
Sep 10 22:06:01.393678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:01.393430    2563 generic.go:356] "Generic (PLEG): container finished" podID="cb4095b3-21dd-4367-af3e-0aced8f43716" containerID="606856214f89237356ad138c94e3f664f74e30634310050aed063969ad37de7d" exitCode=0
Sep 10 22:06:01.393678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:01.393530    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerDied","Data":"606856214f89237356ad138c94e3f664f74e30634310050aed063969ad37de7d"}
Sep 10 22:06:02.104192 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:02.104120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:02.104319 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:02.104231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:02.104319 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:02.104237    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:06:02.104319 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:02.104236    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:02.104412 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:02.104323    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:06:02.104468 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:02.104411    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:06:03.835194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:03.835163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:03.835194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:03.835200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:03.835655 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.835313    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 22:06:03.835655 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.835334    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:06:03.835655 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.835378    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret podName:f5e33414-abf8-4ebc-a2d2-22d60d529098 nodeName:}" failed. No retries permitted until 2026-09-10 22:06:35.835359622 +0000 UTC m=+65.219865044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret") pod "global-pull-secret-syncer-k8cm5" (UID: "f5e33414-abf8-4ebc-a2d2-22d60d529098") : object "kube-system"/"original-pull-secret" not registered
Sep 10 22:06:03.835655 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.835404    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs podName:0055c7c5-046b-47db-97e6-fb1b4349c8af nodeName:}" failed. No retries permitted until 2026-09-10 22:06:35.835386381 +0000 UTC m=+65.219891800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs") pod "network-metrics-daemon-5gzpc" (UID: "0055c7c5-046b-47db-97e6-fb1b4349c8af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 22:06:03.935662 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:03.935613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:03.935803 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.935775    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 22:06:03.935841 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.935803    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 22:06:03.935841 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.935817    2563 projected.go:196] Error preparing data for projected volume kube-api-access-878c9 for pod openshift-network-diagnostics/network-check-target-4tpcc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:06:03.935934 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:03.935884    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9 podName:0a8500ef-406f-4329-acd2-0e6d202399a5 nodeName:}" failed. No retries permitted until 2026-09-10 22:06:35.935863054 +0000 UTC m=+65.320368459 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-878c9" (UniqueName: "kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9") pod "network-check-target-4tpcc" (UID: "0a8500ef-406f-4329-acd2-0e6d202399a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 22:06:04.105221 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:04.105141    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:04.105221 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:04.105163    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:04.105418 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:04.105264    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:04.105418 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:04.105278    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5gzpc" podUID="0055c7c5-046b-47db-97e6-fb1b4349c8af"
Sep 10 22:06:04.105418 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:04.105364    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8cm5" podUID="f5e33414-abf8-4ebc-a2d2-22d60d529098"
Sep 10 22:06:04.105418 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:04.105395    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4tpcc" podUID="0a8500ef-406f-4329-acd2-0e6d202399a5"
Sep 10 22:06:04.860579 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:04.860542    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-25.ec2.internal" event="NodeReady"
Sep 10 22:06:04.861159 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:04.860696    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 22:06:04.985406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:04.985372    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mxg8z"]
Sep 10 22:06:05.000398 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.000325    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2k4zb"]
Sep 10 22:06:05.000565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.000502    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.004520 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.004491    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 22:06:05.004690 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.004498    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mb5j2\""
Sep 10 22:06:05.004690 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.004499    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 22:06:05.009291 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.009229    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mxg8z"]
Sep 10 22:06:05.009291 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.009275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2k4zb"]
Sep 10 22:06:05.009410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.009365    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.011859 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.011836    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 22:06:05.011985 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.011867    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wxdc9\""
Sep 10 22:06:05.012238 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.012219    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 22:06:05.012333 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.012277    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 22:06:05.144182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.144147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk2l6\" (UniqueName: \"kubernetes.io/projected/c6f77c79-5458-4700-81bb-ea60234efad2-kube-api-access-kk2l6\") pod \"ingress-canary-2k4zb\" (UID: \"c6f77c79-5458-4700-81bb-ea60234efad2\") " pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.144182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.144184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gfmw\" (UniqueName: \"kubernetes.io/projected/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-kube-api-access-4gfmw\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.144416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.144218    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-tmp-dir\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.144416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.144290    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-config-volume\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.144416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.144331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6f77c79-5458-4700-81bb-ea60234efad2-cert\") pod \"ingress-canary-2k4zb\" (UID: \"c6f77c79-5458-4700-81bb-ea60234efad2\") " pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.144416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.144350    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.245545 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.245511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk2l6\" (UniqueName: \"kubernetes.io/projected/c6f77c79-5458-4700-81bb-ea60234efad2-kube-api-access-kk2l6\") pod \"ingress-canary-2k4zb\" (UID: \"c6f77c79-5458-4700-81bb-ea60234efad2\") " pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.245545 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.245545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gfmw\" (UniqueName: \"kubernetes.io/projected/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-kube-api-access-4gfmw\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.245786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.245569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-tmp-dir\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.245786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.245700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-config-volume\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.245786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.245759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6f77c79-5458-4700-81bb-ea60234efad2-cert\") pod \"ingress-canary-2k4zb\" (UID: \"c6f77c79-5458-4700-81bb-ea60234efad2\") " pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.245934 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.245785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.245984 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:05.245944    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:06:05.246041 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:05.246011    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls podName:70421c90-6fde-44da-aa1d-cf6b7a01f6dc nodeName:}" failed. No retries permitted until 2026-09-10 22:06:05.745990887 +0000 UTC m=+35.130496291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls") pod "dns-default-mxg8z" (UID: "70421c90-6fde-44da-aa1d-cf6b7a01f6dc") : secret "dns-default-metrics-tls" not found
Sep 10 22:06:05.246116 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.246093    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-tmp-dir\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.246342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.246320    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-config-volume\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.250706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.250655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c6f77c79-5458-4700-81bb-ea60234efad2-cert\") pod \"ingress-canary-2k4zb\" (UID: \"c6f77c79-5458-4700-81bb-ea60234efad2\") " pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.254966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.254938    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk2l6\" (UniqueName: \"kubernetes.io/projected/c6f77c79-5458-4700-81bb-ea60234efad2-kube-api-access-kk2l6\") pod \"ingress-canary-2k4zb\" (UID: \"c6f77c79-5458-4700-81bb-ea60234efad2\") " pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.255181 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.255166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gfmw\" (UniqueName: \"kubernetes.io/projected/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-kube-api-access-4gfmw\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.320358 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.320323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2k4zb"
Sep 10 22:06:05.750807 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:05.750763    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:05.750976 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:05.750929    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:06:05.751025 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:05.751010    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls podName:70421c90-6fde-44da-aa1d-cf6b7a01f6dc nodeName:}" failed. No retries permitted until 2026-09-10 22:06:06.75099444 +0000 UTC m=+36.135499843 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls") pod "dns-default-mxg8z" (UID: "70421c90-6fde-44da-aa1d-cf6b7a01f6dc") : secret "dns-default-metrics-tls" not found
Sep 10 22:06:06.105028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.104955    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:06.105426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.105019    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:06.105426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.105123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:06.110480 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.110454    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 22:06:06.110480 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.110475    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 22:06:06.110650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.110488    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 22:06:06.110650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.110552    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8m774\""
Sep 10 22:06:06.110650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.110574    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dmqhm\""
Sep 10 22:06:06.110747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.110716    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 22:06:06.454205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.454174    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2k4zb"]
Sep 10 22:06:06.458424 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:06.458392    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6f77c79_5458_4700_81bb_ea60234efad2.slice/crio-9103eb3a50dd2345e70f10489b97f4b31f39397850acd5b50a028cbc25c72480 WatchSource:0}: Error finding container 9103eb3a50dd2345e70f10489b97f4b31f39397850acd5b50a028cbc25c72480: Status 404 returned error can't find the container with id 9103eb3a50dd2345e70f10489b97f4b31f39397850acd5b50a028cbc25c72480
Sep 10 22:06:06.758651 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:06.758479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:06.758766 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:06.758633    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:06:06.758808 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:06.758776    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls podName:70421c90-6fde-44da-aa1d-cf6b7a01f6dc nodeName:}" failed. No retries permitted until 2026-09-10 22:06:08.758757034 +0000 UTC m=+38.143262458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls") pod "dns-default-mxg8z" (UID: "70421c90-6fde-44da-aa1d-cf6b7a01f6dc") : secret "dns-default-metrics-tls" not found
Sep 10 22:06:07.410480 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:07.410426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2k4zb" event={"ID":"c6f77c79-5458-4700-81bb-ea60234efad2","Type":"ContainerStarted","Data":"9103eb3a50dd2345e70f10489b97f4b31f39397850acd5b50a028cbc25c72480"}
Sep 10 22:06:07.413952 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:07.413924    2563 generic.go:356] "Generic (PLEG): container finished" podID="cb4095b3-21dd-4367-af3e-0aced8f43716" containerID="5ced88f8a97518cd10a3e850844e9b0cf205b888065714b0051b79d59bc0984a" exitCode=0
Sep 10 22:06:07.414105 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:07.413990    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerDied","Data":"5ced88f8a97518cd10a3e850844e9b0cf205b888065714b0051b79d59bc0984a"}
Sep 10 22:06:08.420189 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:08.420154    2563 generic.go:356] "Generic (PLEG): container finished" podID="cb4095b3-21dd-4367-af3e-0aced8f43716" containerID="bff42dc3319f44c499145e51c04e8c98c0225b68a53009d2418650316ff02ba3" exitCode=0
Sep 10 22:06:08.420869 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:08.420210    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerDied","Data":"bff42dc3319f44c499145e51c04e8c98c0225b68a53009d2418650316ff02ba3"}
Sep 10 22:06:08.772999 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:08.772940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:08.773195 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:08.773135    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:06:08.773261 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:08.773233    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls podName:70421c90-6fde-44da-aa1d-cf6b7a01f6dc nodeName:}" failed. No retries permitted until 2026-09-10 22:06:12.773210313 +0000 UTC m=+42.157715719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls") pod "dns-default-mxg8z" (UID: "70421c90-6fde-44da-aa1d-cf6b7a01f6dc") : secret "dns-default-metrics-tls" not found
Sep 10 22:06:09.426286 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:09.426209    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" event={"ID":"cb4095b3-21dd-4367-af3e-0aced8f43716","Type":"ContainerStarted","Data":"ccfe3fa794ef031b67d9c97dce9fcb66ec2aa7133ab9af036b320ecd0171dccc"}
Sep 10 22:06:09.453776 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:09.453732    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4v5g8" podStartSLOduration=6.514543455 podStartE2EDuration="37.453716888s" totalImagesPullingTime="30.939173433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:09.449467354 +0000 UTC m=+38.833972775" watchObservedRunningTime="2026-09-10 22:06:09.453716888 +0000 UTC m=+38.838222312"
Sep 10 22:06:10.430104 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.430069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2k4zb" event={"ID":"c6f77c79-5458-4700-81bb-ea60234efad2","Type":"ContainerStarted","Data":"073a73f91e538f6942bf952e8c48e68ac20e12be5394edcc2a4a8566a55f5895"}
Sep 10 22:06:10.457372 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.457334    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2k4zb" podStartSLOduration=3.412438299 podStartE2EDuration="6.457319088s" totalImagesPullingTime="3.044880789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:10.454523445 +0000 UTC m=+39.839028867" watchObservedRunningTime="2026-09-10 22:06:10.457319088 +0000 UTC m=+39.841824513"
Sep 10 22:06:10.533164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.533138    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"]
Sep 10 22:06:10.551154 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.551133    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"]
Sep 10 22:06:10.551274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.551222    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"
Sep 10 22:06:10.554718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.554693    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9fzsp\""
Sep 10 22:06:10.554718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.554708    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 22:06:10.556542 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.556523    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 22:06:10.688962 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.688888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkvbh\" (UniqueName: \"kubernetes.io/projected/39502c8f-7404-4c79-87a2-bbe3bf9ab7bf-kube-api-access-jkvbh\") pod \"migrator-8b4d697c5-rt78z\" (UID: \"39502c8f-7404-4c79-87a2-bbe3bf9ab7bf\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"
Sep 10 22:06:10.789758 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.789732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkvbh\" (UniqueName: \"kubernetes.io/projected/39502c8f-7404-4c79-87a2-bbe3bf9ab7bf-kube-api-access-jkvbh\") pod \"migrator-8b4d697c5-rt78z\" (UID: \"39502c8f-7404-4c79-87a2-bbe3bf9ab7bf\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"
Sep 10 22:06:10.798996 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.798965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkvbh\" (UniqueName: \"kubernetes.io/projected/39502c8f-7404-4c79-87a2-bbe3bf9ab7bf-kube-api-access-jkvbh\") pod \"migrator-8b4d697c5-rt78z\" (UID: \"39502c8f-7404-4c79-87a2-bbe3bf9ab7bf\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"
Sep 10 22:06:10.859366 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:10.859341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"
Sep 10 22:06:11.140064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.140031    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z"]
Sep 10 22:06:11.142992 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:11.142967    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39502c8f_7404_4c79_87a2_bbe3bf9ab7bf.slice/crio-98b906f5889bf498db174fba05cdf8f701f12f5eb7caca8c5d285cee6acb178a WatchSource:0}: Error finding container 98b906f5889bf498db174fba05cdf8f701f12f5eb7caca8c5d285cee6acb178a: Status 404 returned error can't find the container with id 98b906f5889bf498db174fba05cdf8f701f12f5eb7caca8c5d285cee6acb178a
Sep 10 22:06:11.424052 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.423984    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pplx7"]
Sep 10 22:06:11.453085 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.453043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z" event={"ID":"39502c8f-7404-4c79-87a2-bbe3bf9ab7bf","Type":"ContainerStarted","Data":"98b906f5889bf498db174fba05cdf8f701f12f5eb7caca8c5d285cee6acb178a"}
Sep 10 22:06:11.453085 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.453083    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pplx7"]
Sep 10 22:06:11.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.453162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.455368 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.455347    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 22:06:11.455609 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.455588    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 22:06:11.456252 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.456231    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 22:06:11.456252 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.456231    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bjqmf\""
Sep 10 22:06:11.456469 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.456454    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 22:06:11.595630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.595595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-data-volume\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.595630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.595625    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rznr\" (UniqueName: \"kubernetes.io/projected/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-api-access-5rznr\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.595846 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.595707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.595883 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.595867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.595940 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.595921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-crio-socket\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.697091 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.697031    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.697091 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.697065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.697251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.697126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-crio-socket\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.697251 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:11.697181    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:11.697251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.697218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-data-volume\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.697251 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:11.697241    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:12.197225094 +0000 UTC m=+41.581730511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:11.697425 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.697267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rznr\" (UniqueName: \"kubernetes.io/projected/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-api-access-5rznr\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.697425 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:11.697290    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:06:11.697425 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.697335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-crio-socket\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.697425 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:11.697358    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:12.197340947 +0000 UTC m=+41.581846375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : configmap "kube-rbac-proxy" not found
Sep 10 22:06:11.697584 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.697545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-data-volume\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:11.707392 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:11.707373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rznr\" (UniqueName: \"kubernetes.io/projected/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-api-access-5rznr\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:12.202166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:12.202128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:12.202348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:12.202176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:12.202348 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:12.202258    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:06:12.202348 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:12.202273    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:12.202348 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:12.202326    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:13.202307976 +0000 UTC m=+42.586813396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : configmap "kube-rbac-proxy" not found
Sep 10 22:06:12.202348 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:12.202342    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:13.202334186 +0000 UTC m=+42.586839589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:12.805454 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:12.805414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:12.805803 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:12.805541    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:06:12.805803 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:12.805594    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls podName:70421c90-6fde-44da-aa1d-cf6b7a01f6dc nodeName:}" failed. No retries permitted until 2026-09-10 22:06:20.805580487 +0000 UTC m=+50.190085890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls") pod "dns-default-mxg8z" (UID: "70421c90-6fde-44da-aa1d-cf6b7a01f6dc") : secret "dns-default-metrics-tls" not found
Sep 10 22:06:13.208665 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.208581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:13.208855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.208696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:13.208855 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:13.208717    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:06:13.208855 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:13.208790    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:15.208771863 +0000 UTC m=+44.593277267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : configmap "kube-rbac-proxy" not found
Sep 10 22:06:13.208855 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:13.208808    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:13.208855 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:13.208849    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:15.208837887 +0000 UTC m=+44.593343289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:13.439823 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.439792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z" event={"ID":"39502c8f-7404-4c79-87a2-bbe3bf9ab7bf","Type":"ContainerStarted","Data":"50cdf95810a6bd63e6b1c3533e2933eddaa1fa80804c9c594585e583a7858d64"}
Sep 10 22:06:13.440014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.439848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z" event={"ID":"39502c8f-7404-4c79-87a2-bbe3bf9ab7bf","Type":"ContainerStarted","Data":"b66a3ab3a5f75ddb08267cc2d1a8b15d6fdca3c6ff6a5eb6810029eb98f8cef6"}
Sep 10 22:06:13.456513 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.456467    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-rt78z" podStartSLOduration=1.782676251 podStartE2EDuration="3.456429575s" totalImagesPullingTime="1.673753324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:13.454754686 +0000 UTC m=+42.839260111" watchObservedRunningTime="2026-09-10 22:06:13.456429575 +0000 UTC m=+42.840934999"
Sep 10 22:06:13.948067 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.948033    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ddm4k"]
Sep 10 22:06:13.972236 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.972213    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ddm4k"]
Sep 10 22:06:13.972372 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.972297    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:13.984526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.984487    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 22:06:13.984526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.984488    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 22:06:13.994981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.994966    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-77qcj_c58b53b6-ef61-400e-b3b4-beaf17da2443/dns-node-resolver/0.log"
Sep 10 22:06:13.995692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.995670    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 22:06:13.995775 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.995675    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 22:06:13.995856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:13.995836    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bhdhz\""
Sep 10 22:06:14.116458 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.116411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-signing-key\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.116619 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.116469    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk7cz\" (UniqueName: \"kubernetes.io/projected/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-kube-api-access-dk7cz\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.116619 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.116523    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-signing-cabundle\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.217972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.217890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk7cz\" (UniqueName: \"kubernetes.io/projected/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-kube-api-access-dk7cz\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.217972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.217935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-signing-cabundle\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.218157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.218017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-signing-key\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.218681 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.218660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-signing-cabundle\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.220492 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.220474    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-signing-key\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.232740 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.232711    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk7cz\" (UniqueName: \"kubernetes.io/projected/a9207e47-ee3a-42e7-86d3-8f22c21f3c5c-kube-api-access-dk7cz\") pod \"service-ca-568869fdc5-ddm4k\" (UID: \"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c\") " pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.280591 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.280565    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-ddm4k"
Sep 10 22:06:14.573860 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:14.573818    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9207e47_ee3a_42e7_86d3_8f22c21f3c5c.slice/crio-d44a26f1e957c37966e500c726d5c6679f4f24e433158df1cfb0d320ff04c746 WatchSource:0}: Error finding container d44a26f1e957c37966e500c726d5c6679f4f24e433158df1cfb0d320ff04c746: Status 404 returned error can't find the container with id d44a26f1e957c37966e500c726d5c6679f4f24e433158df1cfb0d320ff04c746
Sep 10 22:06:14.575206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:14.575182    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ddm4k"]
Sep 10 22:06:15.196876 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:15.196848    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vk5zq_9ddc1a73-dcad-4177-ae60-ed096db3624d/node-ca/0.log"
Sep 10 22:06:15.226340 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:15.226309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:15.226547 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:15.226349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:15.226547 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:15.226510    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:15.226547 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:15.226526    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:06:15.226716 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:15.226580    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:19.226560851 +0000 UTC m=+48.611066256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:15.226716 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:15.226606    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:19.226589369 +0000 UTC m=+48.611094786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : configmap "kube-rbac-proxy" not found
Sep 10 22:06:15.445924 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:15.445886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-ddm4k" event={"ID":"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c","Type":"ContainerStarted","Data":"d44a26f1e957c37966e500c726d5c6679f4f24e433158df1cfb0d320ff04c746"}
Sep 10 22:06:15.797989 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:15.797953    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2k4zb_c6f77c79-5458-4700-81bb-ea60234efad2/serve-healthcheck-canary/0.log"
Sep 10 22:06:16.400240 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:16.400211    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-rt78z_39502c8f-7404-4c79-87a2-bbe3bf9ab7bf/migrator/0.log"
Sep 10 22:06:16.595348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:16.595322    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-rt78z_39502c8f-7404-4c79-87a2-bbe3bf9ab7bf/graceful-termination/0.log"
Sep 10 22:06:17.453429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:17.453349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-ddm4k" event={"ID":"a9207e47-ee3a-42e7-86d3-8f22c21f3c5c","Type":"ContainerStarted","Data":"d0e8491ee8a7248f8b3d8cfd4420a0b5f199fe0c6ce09d5a622167d300337ab9"}
Sep 10 22:06:17.505696 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:17.505646    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-ddm4k" podStartSLOduration=2.274042435 podStartE2EDuration="4.50562928s" totalImagesPullingTime="2.231586845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:17.493063843 +0000 UTC m=+46.877569282" watchObservedRunningTime="2026-09-10 22:06:17.50562928 +0000 UTC m=+46.890134702"
Sep 10 22:06:19.261119 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:19.261081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:19.261550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:19.261117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:19.261550 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:19.261221    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 22:06:19.261550 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:19.261223    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:19.261550 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:19.261272    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:27.26125847 +0000 UTC m=+56.645763873 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : configmap "kube-rbac-proxy" not found
Sep 10 22:06:19.261550 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:19.261285    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:27.261278982 +0000 UTC m=+56.645784385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:20.874714 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:20.874682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:20.875213 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:20.874802    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 22:06:20.875213 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:20.874856    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls podName:70421c90-6fde-44da-aa1d-cf6b7a01f6dc nodeName:}" failed. No retries permitted until 2026-09-10 22:06:36.874843188 +0000 UTC m=+66.259348590 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls") pod "dns-default-mxg8z" (UID: "70421c90-6fde-44da-aa1d-cf6b7a01f6dc") : secret "dns-default-metrics-tls" not found
Sep 10 22:06:27.317623 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:27.317589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:27.317623 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:27.317628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:27.318175 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:27.317742    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:27.318175 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:06:27.317813    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls podName:3e3d07f6-baf1-45e6-919c-12bf63bf28ff nodeName:}" failed. No retries permitted until 2026-09-10 22:06:43.317797096 +0000 UTC m=+72.702302499 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pplx7" (UID: "3e3d07f6-baf1-45e6-919c-12bf63bf28ff") : secret "insights-runtime-extractor-tls" not found
Sep 10 22:06:27.318175 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:27.318087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:30.404507 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:30.404478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h452"
Sep 10 22:06:35.877125 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.877084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:35.877125 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.877128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:35.890030 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.890009    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 22:06:35.890111 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.890009    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 22:06:35.900577 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.900549    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e33414-abf8-4ebc-a2d2-22d60d529098-original-pull-secret\") pod \"global-pull-secret-syncer-k8cm5\" (UID: \"f5e33414-abf8-4ebc-a2d2-22d60d529098\") " pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:35.900577 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.900567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0055c7c5-046b-47db-97e6-fb1b4349c8af-metrics-certs\") pod \"network-metrics-daemon-5gzpc\" (UID: \"0055c7c5-046b-47db-97e6-fb1b4349c8af\") " pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:35.977945 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.977898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:35.985524 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.985503    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 22:06:35.996222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:35.996203    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 22:06:36.001749 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.001728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-878c9\" (UniqueName: \"kubernetes.io/projected/0a8500ef-406f-4329-acd2-0e6d202399a5-kube-api-access-878c9\") pod \"network-check-target-4tpcc\" (UID: \"0a8500ef-406f-4329-acd2-0e6d202399a5\") " pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:36.120322 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.120291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8cm5"
Sep 10 22:06:36.125172 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.125152    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8m774\""
Sep 10 22:06:36.125273 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.125208    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5gzpc"
Sep 10 22:06:36.132851 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.132811    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dmqhm\""
Sep 10 22:06:36.135461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.135448    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:36.430102 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.429959    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k8cm5"]
Sep 10 22:06:36.455330 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:36.455306    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0055c7c5_046b_47db_97e6_fb1b4349c8af.slice/crio-b3fe6f5fbcafd0eb1338db7ae944c400ff66a08a4e29ac0fa12c7fee750b071b WatchSource:0}: Error finding container b3fe6f5fbcafd0eb1338db7ae944c400ff66a08a4e29ac0fa12c7fee750b071b: Status 404 returned error can't find the container with id b3fe6f5fbcafd0eb1338db7ae944c400ff66a08a4e29ac0fa12c7fee750b071b
Sep 10 22:06:36.458149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.458125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5gzpc"]
Sep 10 22:06:36.475778 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:36.475756    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a8500ef_406f_4329_acd2_0e6d202399a5.slice/crio-7d67c309039bd9c2af30a7ac7b81077daef274ea4c9efb9ffc02c17ee17349be WatchSource:0}: Error finding container 7d67c309039bd9c2af30a7ac7b81077daef274ea4c9efb9ffc02c17ee17349be: Status 404 returned error can't find the container with id 7d67c309039bd9c2af30a7ac7b81077daef274ea4c9efb9ffc02c17ee17349be
Sep 10 22:06:36.479945 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.479926    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4tpcc"]
Sep 10 22:06:36.501219 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.501178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4tpcc" event={"ID":"0a8500ef-406f-4329-acd2-0e6d202399a5","Type":"ContainerStarted","Data":"7d67c309039bd9c2af30a7ac7b81077daef274ea4c9efb9ffc02c17ee17349be"}
Sep 10 22:06:36.502327 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.502309    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5gzpc" event={"ID":"0055c7c5-046b-47db-97e6-fb1b4349c8af","Type":"ContainerStarted","Data":"b3fe6f5fbcafd0eb1338db7ae944c400ff66a08a4e29ac0fa12c7fee750b071b"}
Sep 10 22:06:36.503419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.503392    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k8cm5" event={"ID":"f5e33414-abf8-4ebc-a2d2-22d60d529098","Type":"ContainerStarted","Data":"9e1109b6a744838b6be640cc0022caa83180784873ad6d7242b93daf378dae29"}
Sep 10 22:06:36.882790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.882762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:36.885308 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.885279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70421c90-6fde-44da-aa1d-cf6b7a01f6dc-metrics-tls\") pod \"dns-default-mxg8z\" (UID: \"70421c90-6fde-44da-aa1d-cf6b7a01f6dc\") " pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:36.919084 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.919050    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"]
Sep 10 22:06:36.932329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.932304    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"]
Sep 10 22:06:36.932505 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.932471    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:36.947859 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.947833    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 22:06:36.947990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.947833    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 22:06:36.947990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.947879    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 22:06:36.948091 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.948077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"]
Sep 10 22:06:36.948126 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.948101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"]
Sep 10 22:06:36.948201 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.948188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:36.948599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.948579    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 22:06:36.960238 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.960217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 22:06:36.960363 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.960251    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 22:06:36.960363 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.960217    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 22:06:36.974983 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.974962    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gq7r5\""
Sep 10 22:06:36.975158 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.975142    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 22:06:36.983664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:36.983772 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz2zc\" (UniqueName: \"kubernetes.io/projected/64e0016c-82a2-4ea3-b574-653f25faa1d8-kube-api-access-dz2zc\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:36.983836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983779    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e0016c-82a2-4ea3-b574-653f25faa1d8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:36.983836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:36.983936 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983838    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bgg5\" (UniqueName: \"kubernetes.io/projected/56453de7-6f1d-4833-a550-b9d4e677eb18-kube-api-access-6bgg5\") pod \"managed-serviceaccount-addon-agent-5898757b58-scjgh\" (UID: \"56453de7-6f1d-4833-a550-b9d4e677eb18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:36.983936 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-ca\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:36.983936 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-hub\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:36.984082 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:36.983927    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56453de7-6f1d-4833-a550-b9d4e677eb18-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5898757b58-scjgh\" (UID: \"56453de7-6f1d-4833-a550-b9d4e677eb18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:37.084629 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56453de7-6f1d-4833-a550-b9d4e677eb18-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5898757b58-scjgh\" (UID: \"56453de7-6f1d-4833-a550-b9d4e677eb18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:37.084838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.084838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz2zc\" (UniqueName: \"kubernetes.io/projected/64e0016c-82a2-4ea3-b574-653f25faa1d8-kube-api-access-dz2zc\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.084838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084731    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e0016c-82a2-4ea3-b574-653f25faa1d8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.084838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.084838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bgg5\" (UniqueName: \"kubernetes.io/projected/56453de7-6f1d-4833-a550-b9d4e677eb18-kube-api-access-6bgg5\") pod \"managed-serviceaccount-addon-agent-5898757b58-scjgh\" (UID: \"56453de7-6f1d-4833-a550-b9d4e677eb18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:37.084838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-ca\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.084838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.084827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-hub\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.087431 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.085946    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/64e0016c-82a2-4ea3-b574-653f25faa1d8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.089238 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.089208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-hub\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.090553 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.090507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/56453de7-6f1d-4833-a550-b9d4e677eb18-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5898757b58-scjgh\" (UID: \"56453de7-6f1d-4833-a550-b9d4e677eb18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:37.090671 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.090592    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.092773 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.092731    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-ca\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.096580 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.096490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/64e0016c-82a2-4ea3-b574-653f25faa1d8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.121408 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.121340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz2zc\" (UniqueName: \"kubernetes.io/projected/64e0016c-82a2-4ea3-b574-653f25faa1d8-kube-api-access-dz2zc\") pod \"cluster-proxy-proxy-agent-6c8984c9d6-48gvq\" (UID: \"64e0016c-82a2-4ea3-b574-653f25faa1d8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.129677 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.129620    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mb5j2\""
Sep 10 22:06:37.131912 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.131864    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bgg5\" (UniqueName: \"kubernetes.io/projected/56453de7-6f1d-4833-a550-b9d4e677eb18-kube-api-access-6bgg5\") pod \"managed-serviceaccount-addon-agent-5898757b58-scjgh\" (UID: \"56453de7-6f1d-4833-a550-b9d4e677eb18\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:37.134192 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.133927    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:37.250478 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.250425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"
Sep 10 22:06:37.258953 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.258909    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"
Sep 10 22:06:37.441309 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.441280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mxg8z"]
Sep 10 22:06:37.554456 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.554379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq"]
Sep 10 22:06:37.579369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:37.579337    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh"]
Sep 10 22:06:37.666329 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:37.666290    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70421c90_6fde_44da_aa1d_cf6b7a01f6dc.slice/crio-bc9d9064553afb1320f88602aefc8b994d07dbc67cfebf69439533554e28ec6d WatchSource:0}: Error finding container bc9d9064553afb1320f88602aefc8b994d07dbc67cfebf69439533554e28ec6d: Status 404 returned error can't find the container with id bc9d9064553afb1320f88602aefc8b994d07dbc67cfebf69439533554e28ec6d
Sep 10 22:06:37.667919 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:37.667890    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64e0016c_82a2_4ea3_b574_653f25faa1d8.slice/crio-996ce05fe3f47a76c393717afec35526acfe4a5adce95a94620bff9452e1f603 WatchSource:0}: Error finding container 996ce05fe3f47a76c393717afec35526acfe4a5adce95a94620bff9452e1f603: Status 404 returned error can't find the container with id 996ce05fe3f47a76c393717afec35526acfe4a5adce95a94620bff9452e1f603
Sep 10 22:06:37.668535 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:37.668506    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56453de7_6f1d_4833_a550_b9d4e677eb18.slice/crio-96ab1063fb05fc0da7a1856daaace2585f2aa14f03a4f43617360cf47fc45df4 WatchSource:0}: Error finding container 96ab1063fb05fc0da7a1856daaace2585f2aa14f03a4f43617360cf47fc45df4: Status 404 returned error can't find the container with id 96ab1063fb05fc0da7a1856daaace2585f2aa14f03a4f43617360cf47fc45df4
Sep 10 22:06:38.517491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:38.516707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5gzpc" event={"ID":"0055c7c5-046b-47db-97e6-fb1b4349c8af","Type":"ContainerStarted","Data":"e568336445722041c27b8c707b3d629f38e1a24ab2d870b8ab4d458805aa000b"}
Sep 10 22:06:38.517491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:38.516816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5gzpc" event={"ID":"0055c7c5-046b-47db-97e6-fb1b4349c8af","Type":"ContainerStarted","Data":"d6029f7b3f79cd7a61a49dd6cb001dd5fb3044b88c5e57cdbf156fd54475cc7f"}
Sep 10 22:06:38.520002 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:38.519922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mxg8z" event={"ID":"70421c90-6fde-44da-aa1d-cf6b7a01f6dc","Type":"ContainerStarted","Data":"bc9d9064553afb1320f88602aefc8b994d07dbc67cfebf69439533554e28ec6d"}
Sep 10 22:06:38.522305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:38.522247    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh" event={"ID":"56453de7-6f1d-4833-a550-b9d4e677eb18","Type":"ContainerStarted","Data":"96ab1063fb05fc0da7a1856daaace2585f2aa14f03a4f43617360cf47fc45df4"}
Sep 10 22:06:38.524920 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:38.524876    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq" event={"ID":"64e0016c-82a2-4ea3-b574-653f25faa1d8","Type":"ContainerStarted","Data":"996ce05fe3f47a76c393717afec35526acfe4a5adce95a94620bff9452e1f603"}
Sep 10 22:06:38.569615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:38.569559    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5gzpc" podStartSLOduration=65.290688231 podStartE2EDuration="1m6.569535605s" totalImagesPullingTime="1.278847374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:38.568342986 +0000 UTC m=+67.952848413" watchObservedRunningTime="2026-09-10 22:06:38.569535605 +0000 UTC m=+67.954041030"
Sep 10 22:06:43.334844 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:43.334808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:43.337174 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:43.337145    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e3d07f6-baf1-45e6-919c-12bf63bf28ff-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pplx7\" (UID: \"3e3d07f6-baf1-45e6-919c-12bf63bf28ff\") " pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:43.576296 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:43.576260    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bjqmf\""
Sep 10 22:06:43.582364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:43.582345    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pplx7"
Sep 10 22:06:45.093476 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:45.093418    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3d07f6_baf1_45e6_919c_12bf63bf28ff.slice/crio-824584175b2c0e4792ae2cfef99926b8c8db2e8edb21843da9a154e1e3954548 WatchSource:0}: Error finding container 824584175b2c0e4792ae2cfef99926b8c8db2e8edb21843da9a154e1e3954548: Status 404 returned error can't find the container with id 824584175b2c0e4792ae2cfef99926b8c8db2e8edb21843da9a154e1e3954548
Sep 10 22:06:45.093977 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.093956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pplx7"]
Sep 10 22:06:45.555025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.554977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pplx7" event={"ID":"3e3d07f6-baf1-45e6-919c-12bf63bf28ff","Type":"ContainerStarted","Data":"a3a2de5b29e0193b8884bff4a0eafb26f5badcbfb42d0ece80a5a857f02605e2"}
Sep 10 22:06:45.555193 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.555061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pplx7" event={"ID":"3e3d07f6-baf1-45e6-919c-12bf63bf28ff","Type":"ContainerStarted","Data":"824584175b2c0e4792ae2cfef99926b8c8db2e8edb21843da9a154e1e3954548"}
Sep 10 22:06:45.556731 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.556705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh" event={"ID":"56453de7-6f1d-4833-a550-b9d4e677eb18","Type":"ContainerStarted","Data":"7486003cc97e409f899ffc03f90120586cabc55d830a5bf42df68ca71bfa3523"}
Sep 10 22:06:45.559106 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.559077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq" event={"ID":"64e0016c-82a2-4ea3-b574-653f25faa1d8","Type":"ContainerStarted","Data":"d18270e12156747ae6816169bc91b95d0eee7df2bea8f286acac763c4fb37fca"}
Sep 10 22:06:45.559241 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.559140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq" event={"ID":"64e0016c-82a2-4ea3-b574-653f25faa1d8","Type":"ContainerStarted","Data":"e797d01cce94542efa9503c281d9aa31e251d1e0faf10fdec310a9f63c1196c1"}
Sep 10 22:06:45.559241 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.559161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq" event={"ID":"64e0016c-82a2-4ea3-b574-653f25faa1d8","Type":"ContainerStarted","Data":"f6e002b1b474eb021ee1bcbd8b965755bbf089264cf472da9771573cf11a1b1e"}
Sep 10 22:06:45.560667 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.560640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4tpcc" event={"ID":"0a8500ef-406f-4329-acd2-0e6d202399a5","Type":"ContainerStarted","Data":"484bece6497a6cd147da68d8c2a08461ca17a420d2b775fef961c77d90fb7754"}
Sep 10 22:06:45.560782 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.560706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:06:45.562342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.562315    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k8cm5" event={"ID":"f5e33414-abf8-4ebc-a2d2-22d60d529098","Type":"ContainerStarted","Data":"3950956d5cd2709515f53d47de246ce18cd00ab38b11f17c4f2e27c6ab8df15e"}
Sep 10 22:06:45.564194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.564172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mxg8z" event={"ID":"70421c90-6fde-44da-aa1d-cf6b7a01f6dc","Type":"ContainerStarted","Data":"19ba545db491d57ed535b4a14f174d989a5b7766cc8d6f540ae664d2b917071d"}
Sep 10 22:06:45.564312 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.564208    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mxg8z" event={"ID":"70421c90-6fde-44da-aa1d-cf6b7a01f6dc","Type":"ContainerStarted","Data":"7c302e06d1ae0c32bd79b553434cc1e1dad2a2a76582d8dbc9dc2a5bcaea032b"}
Sep 10 22:06:45.564373 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.564327    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:06:45.610901 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.610853    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5898757b58-scjgh" podStartSLOduration=2.464613845 podStartE2EDuration="9.610840417s" totalImagesPullingTime="7.146226572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:45.610556732 +0000 UTC m=+74.995062157" watchObservedRunningTime="2026-09-10 22:06:45.610840417 +0000 UTC m=+74.995345836"
Sep 10 22:06:45.650990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.650951    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4tpcc" podStartSLOduration=65.313087568 podStartE2EDuration="1m13.650937s" totalImagesPullingTime="8.337849432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:45.650843572 +0000 UTC m=+75.035348996" watchObservedRunningTime="2026-09-10 22:06:45.650937 +0000 UTC m=+75.035442424"
Sep 10 22:06:45.694432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.694390    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mxg8z" podStartSLOduration=34.54860259 podStartE2EDuration="41.694374408s" totalImagesPullingTime="7.145771818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:45.694272245 +0000 UTC m=+75.078777671" watchObservedRunningTime="2026-09-10 22:06:45.694374408 +0000 UTC m=+75.078879832"
Sep 10 22:06:45.752273 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.752225    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k8cm5" podStartSLOduration=65.367296942 podStartE2EDuration="1m13.752209914s" totalImagesPullingTime="8.384912972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:45.751671052 +0000 UTC m=+75.136176477" watchObservedRunningTime="2026-09-10 22:06:45.752209914 +0000 UTC m=+75.136715339"
Sep 10 22:06:45.799960 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:45.799917    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c8984c9d6-48gvq" podStartSLOduration=2.633235549 podStartE2EDuration="9.799902325s" totalImagesPullingTime="7.166666776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:45.798593459 +0000 UTC m=+75.183098884" watchObservedRunningTime="2026-09-10 22:06:45.799902325 +0000 UTC m=+75.184407751"
Sep 10 22:06:46.571477 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:46.571417    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pplx7" event={"ID":"3e3d07f6-baf1-45e6-919c-12bf63bf28ff","Type":"ContainerStarted","Data":"77edaeeb6b277416a878555db6e6ed20c0f183ee7c72f88eedfc63c44a7d589a"}
Sep 10 22:06:47.576510 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:47.576470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pplx7" event={"ID":"3e3d07f6-baf1-45e6-919c-12bf63bf28ff","Type":"ContainerStarted","Data":"758e87f1a988f179a51c81df76c7957e2264382080987356c70a9bef27e9cd5f"}
Sep 10 22:06:47.610288 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:47.610242    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pplx7" podStartSLOduration=34.843428759 podStartE2EDuration="36.61022576s" totalImagesPullingTime="1.766797001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:06:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:47.609746543 +0000 UTC m=+76.994251967" watchObservedRunningTime="2026-09-10 22:06:47.61022576 +0000 UTC m=+76.994731184"
Sep 10 22:06:52.150657 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.150513    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b86j5"]
Sep 10 22:06:52.153749 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.153725    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.177632 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.177604    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 22:06:52.177757 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.177694    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 22:06:52.195365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.195339    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 22:06:52.195494 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.195341    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 22:06:52.195737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.195718    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x7rt7\""
Sep 10 22:06:52.195814 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.195769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 22:06:52.195814 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.195787    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 22:06:52.198922 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.198902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-tls\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.198932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-wtmp\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.198952    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-root\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.198969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199099 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.199049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvctj\" (UniqueName: \"kubernetes.io/projected/dab3fe50-4a30-4571-ab66-fc8463d9276d-kube-api-access-fvctj\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.199095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dab3fe50-4a30-4571-ab66-fc8463d9276d-metrics-client-ca\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.199118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199193 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.199141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-sys\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.199193 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.199155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-textfile\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299593 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299593 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-sys\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299803 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299612    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-textfile\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299803 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-tls\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299803 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299692    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-sys\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299803 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-wtmp\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299803 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-root\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.299803 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.300087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvctj\" (UniqueName: \"kubernetes.io/projected/dab3fe50-4a30-4571-ab66-fc8463d9276d-kube-api-access-fvctj\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.300087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-root\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.300087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299875    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dab3fe50-4a30-4571-ab66-fc8463d9276d-metrics-client-ca\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.300087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-wtmp\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.300087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.299961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-textfile\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.300324 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.300303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.300415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.300395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dab3fe50-4a30-4571-ab66-fc8463d9276d-metrics-client-ca\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.302294 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.302272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.302365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.302313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dab3fe50-4a30-4571-ab66-fc8463d9276d-node-exporter-tls\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.328239 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.328220    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvctj\" (UniqueName: \"kubernetes.io/projected/dab3fe50-4a30-4571-ab66-fc8463d9276d-kube-api-access-fvctj\") pod \"node-exporter-b86j5\" (UID: \"dab3fe50-4a30-4571-ab66-fc8463d9276d\") " pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.462842 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.462715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b86j5"
Sep 10 22:06:52.476157 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:06:52.476123    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab3fe50_4a30_4571_ab66_fc8463d9276d.slice/crio-c76559f6eac96142d8d4ee6f536d75afbf9d2175adbe0faf7510516a82bddce7 WatchSource:0}: Error finding container c76559f6eac96142d8d4ee6f536d75afbf9d2175adbe0faf7510516a82bddce7: Status 404 returned error can't find the container with id c76559f6eac96142d8d4ee6f536d75afbf9d2175adbe0faf7510516a82bddce7
Sep 10 22:06:52.594780 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:52.594740    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b86j5" event={"ID":"dab3fe50-4a30-4571-ab66-fc8463d9276d","Type":"ContainerStarted","Data":"c76559f6eac96142d8d4ee6f536d75afbf9d2175adbe0faf7510516a82bddce7"}
Sep 10 22:06:53.600055 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:53.599982    2563 generic.go:356] "Generic (PLEG): container finished" podID="dab3fe50-4a30-4571-ab66-fc8463d9276d" containerID="c8eca8737e7489e2a3d35eac9d2b7f1a0b2b758542850d8e69fdc1343053754e" exitCode=0
Sep 10 22:06:53.600391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:53.600058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b86j5" event={"ID":"dab3fe50-4a30-4571-ab66-fc8463d9276d","Type":"ContainerDied","Data":"c8eca8737e7489e2a3d35eac9d2b7f1a0b2b758542850d8e69fdc1343053754e"}
Sep 10 22:06:54.605331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:54.605292    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b86j5" event={"ID":"dab3fe50-4a30-4571-ab66-fc8463d9276d","Type":"ContainerStarted","Data":"b16684cd24e2e32c9bd8b73a226b6cfe25aa9b877dc0ba7052404efc4bc80013"}
Sep 10 22:06:54.605707 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:54.605390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b86j5" event={"ID":"dab3fe50-4a30-4571-ab66-fc8463d9276d","Type":"ContainerStarted","Data":"904756877af23767e120b489063b2a6fab9e5098f50d7405fd067bd169880cce"}
Sep 10 22:06:54.655917 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:54.655872    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b86j5" podStartSLOduration=1.751120814 podStartE2EDuration="2.655856613s" totalImagesPullingTime="837.526515ms" totalInitContainerRuntime="67.209284ms" isStatefulPod=true podCreationTimestamp="2026-09-10 22:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:06:54.655192755 +0000 UTC m=+84.039698180" watchObservedRunningTime="2026-09-10 22:06:54.655856613 +0000 UTC m=+84.040362356"
Sep 10 22:06:55.574899 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:06:55.574874    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mxg8z"
Sep 10 22:07:16.574182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:07:16.574143    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4tpcc"
Sep 10 22:10:30.994835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:10:30.994805    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 22:11:04.179182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.179146    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bb5tq"]
Sep 10 22:11:04.181037 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.181022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.183291 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.183261    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 22:11:04.184007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.183987    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-4qfzh\""
Sep 10 22:11:04.184103 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.184025    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 22:11:04.190596 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.190576    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bb5tq"]
Sep 10 22:11:04.266507 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.266473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvdrd\" (UniqueName: \"kubernetes.io/projected/b2dc4bda-818c-4830-8fc8-e69900fc1c6f-kube-api-access-qvdrd\") pod \"cert-manager-webhook-6486ffd997-bb5tq\" (UID: \"b2dc4bda-818c-4830-8fc8-e69900fc1c6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.266507 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.266513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2dc4bda-818c-4830-8fc8-e69900fc1c6f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bb5tq\" (UID: \"b2dc4bda-818c-4830-8fc8-e69900fc1c6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.367754 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.367723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvdrd\" (UniqueName: \"kubernetes.io/projected/b2dc4bda-818c-4830-8fc8-e69900fc1c6f-kube-api-access-qvdrd\") pod \"cert-manager-webhook-6486ffd997-bb5tq\" (UID: \"b2dc4bda-818c-4830-8fc8-e69900fc1c6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.367882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.367759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2dc4bda-818c-4830-8fc8-e69900fc1c6f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bb5tq\" (UID: \"b2dc4bda-818c-4830-8fc8-e69900fc1c6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.375376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.375347    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b2dc4bda-818c-4830-8fc8-e69900fc1c6f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bb5tq\" (UID: \"b2dc4bda-818c-4830-8fc8-e69900fc1c6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.375658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.375635    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvdrd\" (UniqueName: \"kubernetes.io/projected/b2dc4bda-818c-4830-8fc8-e69900fc1c6f-kube-api-access-qvdrd\") pod \"cert-manager-webhook-6486ffd997-bb5tq\" (UID: \"b2dc4bda-818c-4830-8fc8-e69900fc1c6f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.489576 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.489547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:04.756085 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.756017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bb5tq"]
Sep 10 22:11:04.759348 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:11:04.759310    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2dc4bda_818c_4830_8fc8_e69900fc1c6f.slice/crio-4ba2f813dda61b866cfaead4173619131a2a27c38ffa6dd9827a6a41359a1b79 WatchSource:0}: Error finding container 4ba2f813dda61b866cfaead4173619131a2a27c38ffa6dd9827a6a41359a1b79: Status 404 returned error can't find the container with id 4ba2f813dda61b866cfaead4173619131a2a27c38ffa6dd9827a6a41359a1b79
Sep 10 22:11:04.761406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:04.761387    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 22:11:05.419118 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:05.419087    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq" event={"ID":"b2dc4bda-818c-4830-8fc8-e69900fc1c6f","Type":"ContainerStarted","Data":"4ba2f813dda61b866cfaead4173619131a2a27c38ffa6dd9827a6a41359a1b79"}
Sep 10 22:11:08.431420 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:08.431388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq" event={"ID":"b2dc4bda-818c-4830-8fc8-e69900fc1c6f","Type":"ContainerStarted","Data":"e017734569a1958be7b5b6ceb510651ad426aca22488f2ff281597ef0c827d3a"}
Sep 10 22:11:08.446161 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:08.446113    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq" podStartSLOduration=1.475954866 podStartE2EDuration="4.446096794s" totalImagesPullingTime="2.970141928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:11:08.445587159 +0000 UTC m=+337.830092585" watchObservedRunningTime="2026-09-10 22:11:08.446096794 +0000 UTC m=+337.830602218"
Sep 10 22:11:09.434566 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:09.434539    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:15.439900 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:15.439866    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bb5tq"
Sep 10 22:11:17.344622 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.344589    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"]
Sep 10 22:11:17.346578 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.346559    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.349128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.349110    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 22:11:17.349505 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.349490    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 22:11:17.350646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.350634    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2fwsj\""
Sep 10 22:11:17.365039 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.365014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"]
Sep 10 22:11:17.462493 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.462428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jdjg\" (UniqueName: \"kubernetes.io/projected/f1c52885-2908-449a-adca-6ff32128f87b-kube-api-access-7jdjg\") pod \"openshift-lws-operator-fd8ccff4c-pmjpv\" (UID: \"f1c52885-2908-449a-adca-6ff32128f87b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.462639 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.462544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1c52885-2908-449a-adca-6ff32128f87b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pmjpv\" (UID: \"f1c52885-2908-449a-adca-6ff32128f87b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.563370 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.563324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1c52885-2908-449a-adca-6ff32128f87b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pmjpv\" (UID: \"f1c52885-2908-449a-adca-6ff32128f87b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.563570 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.563398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jdjg\" (UniqueName: \"kubernetes.io/projected/f1c52885-2908-449a-adca-6ff32128f87b-kube-api-access-7jdjg\") pod \"openshift-lws-operator-fd8ccff4c-pmjpv\" (UID: \"f1c52885-2908-449a-adca-6ff32128f87b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.563730 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.563710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f1c52885-2908-449a-adca-6ff32128f87b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pmjpv\" (UID: \"f1c52885-2908-449a-adca-6ff32128f87b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.572485 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.572461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jdjg\" (UniqueName: \"kubernetes.io/projected/f1c52885-2908-449a-adca-6ff32128f87b-kube-api-access-7jdjg\") pod \"openshift-lws-operator-fd8ccff4c-pmjpv\" (UID: \"f1c52885-2908-449a-adca-6ff32128f87b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.655421 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.655358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"
Sep 10 22:11:17.923847 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:17.923778    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv"]
Sep 10 22:11:17.926820 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:11:17.926791    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1c52885_2908_449a_adca_6ff32128f87b.slice/crio-03350b2f100ef9c3853832f07cd451d00825425836ab478e57c99e9e41c789ed WatchSource:0}: Error finding container 03350b2f100ef9c3853832f07cd451d00825425836ab478e57c99e9e41c789ed: Status 404 returned error can't find the container with id 03350b2f100ef9c3853832f07cd451d00825425836ab478e57c99e9e41c789ed
Sep 10 22:11:18.472735 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:18.472705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv" event={"ID":"f1c52885-2908-449a-adca-6ff32128f87b","Type":"ContainerStarted","Data":"03350b2f100ef9c3853832f07cd451d00825425836ab478e57c99e9e41c789ed"}
Sep 10 22:11:20.483133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:20.483102    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv" event={"ID":"f1c52885-2908-449a-adca-6ff32128f87b","Type":"ContainerStarted","Data":"85aa3a90deec26ee4c70a921881926444e29e88d64825f3af795a21a79144479"}
Sep 10 22:11:20.499715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:20.499639    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pmjpv" podStartSLOduration=1.0481716699999999 podStartE2EDuration="3.499620641s" totalImagesPullingTime="2.451448971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:11:20.499171661 +0000 UTC m=+349.883677086" watchObservedRunningTime="2026-09-10 22:11:20.499620641 +0000 UTC m=+349.884126067"
Sep 10 22:11:35.985315 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:35.985241    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"]
Sep 10 22:11:35.988361 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:35.988342    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:35.990733 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:35.990706    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 22:11:35.990853 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:35.990741    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 22:11:35.990853 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:35.990745    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 22:11:35.990853 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:35.990749    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4v29l\""
Sep 10 22:11:35.990853 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:35.990745    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 22:11:36.000208 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.000188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"]
Sep 10 22:11:36.091646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.091614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/14da3c6f-0401-40ec-b7b4-3371607fc5ff-manifests\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.091646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.091648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnlzg\" (UniqueName: \"kubernetes.io/projected/14da3c6f-0401-40ec-b7b4-3371607fc5ff-kube-api-access-xnlzg\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.091839 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.091676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14da3c6f-0401-40ec-b7b4-3371607fc5ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.091839 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.091781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14da3c6f-0401-40ec-b7b4-3371607fc5ff-tmp\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.091839 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.091812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/14da3c6f-0401-40ec-b7b4-3371607fc5ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.193053 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.193015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14da3c6f-0401-40ec-b7b4-3371607fc5ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.193229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.193062    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14da3c6f-0401-40ec-b7b4-3371607fc5ff-tmp\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.193229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.193090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/14da3c6f-0401-40ec-b7b4-3371607fc5ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.193229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.193147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/14da3c6f-0401-40ec-b7b4-3371607fc5ff-manifests\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.193229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.193169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnlzg\" (UniqueName: \"kubernetes.io/projected/14da3c6f-0401-40ec-b7b4-3371607fc5ff-kube-api-access-xnlzg\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.193633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.193603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/14da3c6f-0401-40ec-b7b4-3371607fc5ff-manifests\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.195587 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.195564    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14da3c6f-0401-40ec-b7b4-3371607fc5ff-tmp\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.195702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.195657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/14da3c6f-0401-40ec-b7b4-3371607fc5ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.195702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.195663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/14da3c6f-0401-40ec-b7b4-3371607fc5ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.201329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.201308    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnlzg\" (UniqueName: \"kubernetes.io/projected/14da3c6f-0401-40ec-b7b4-3371607fc5ff-kube-api-access-xnlzg\") pod \"opendatahub-operator-controller-manager-995c55587-gx5g8\" (UID: \"14da3c6f-0401-40ec-b7b4-3371607fc5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.298796 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.298717    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:36.574889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:36.574820    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"]
Sep 10 22:11:36.578545 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:11:36.578516    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14da3c6f_0401_40ec_b7b4_3371607fc5ff.slice/crio-6c9a0719f7029b766cfe7900ce96ada759bda1b126b97cef5fdfd5eab54d0b86 WatchSource:0}: Error finding container 6c9a0719f7029b766cfe7900ce96ada759bda1b126b97cef5fdfd5eab54d0b86: Status 404 returned error can't find the container with id 6c9a0719f7029b766cfe7900ce96ada759bda1b126b97cef5fdfd5eab54d0b86
Sep 10 22:11:37.542722 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:37.542681    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8" event={"ID":"14da3c6f-0401-40ec-b7b4-3371607fc5ff","Type":"ContainerStarted","Data":"6c9a0719f7029b766cfe7900ce96ada759bda1b126b97cef5fdfd5eab54d0b86"}
Sep 10 22:11:39.552041 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:39.552006    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8" event={"ID":"14da3c6f-0401-40ec-b7b4-3371607fc5ff","Type":"ContainerStarted","Data":"96242e4b9ee38940fe32e30a592c237bf40b42de3df5210fe2f7a22dd29fdb15"}
Sep 10 22:11:41.561277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:41.561242    2563 generic.go:356] "Generic (PLEG): container finished" podID="14da3c6f-0401-40ec-b7b4-3371607fc5ff" containerID="96242e4b9ee38940fe32e30a592c237bf40b42de3df5210fe2f7a22dd29fdb15" exitCode=0
Sep 10 22:11:41.561754 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:41.561321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8" event={"ID":"14da3c6f-0401-40ec-b7b4-3371607fc5ff","Type":"ContainerDied","Data":"96242e4b9ee38940fe32e30a592c237bf40b42de3df5210fe2f7a22dd29fdb15"}
Sep 10 22:11:42.566802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:42.566767    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8" event={"ID":"14da3c6f-0401-40ec-b7b4-3371607fc5ff","Type":"ContainerStarted","Data":"0de3d05f49c717421c5847c30a5286fa705a360c8e6bd9e809b74f5ce2c58ba8"}
Sep 10 22:11:42.567204 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:42.567008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:11:42.590745 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:42.590700    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8" podStartSLOduration=3.279865843 podStartE2EDuration="7.590686436s" totalImagesPullingTime="3.229377161s" totalInitContainerRuntime="1.081443432s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:11:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:11:42.588818873 +0000 UTC m=+371.973324297" watchObservedRunningTime="2026-09-10 22:11:42.590686436 +0000 UTC m=+371.975191861"
Sep 10 22:11:53.572868 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:11:53.572835    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-995c55587-gx5g8"
Sep 10 22:12:06.080874 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.080838    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"]
Sep 10 22:12:06.085421 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.085402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.088831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.088811    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 22:12:06.089061 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.089038    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 22:12:06.089203 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.089181    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 22:12:06.089275 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.089226    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 22:12:06.096784 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.096762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"]
Sep 10 22:12:06.211504 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.211472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/971ec54a-9d87-48ee-9465-b7f3d685822c-tmp\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.211672 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.211549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/971ec54a-9d87-48ee-9465-b7f3d685822c-tls-certs\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.211672 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.211574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dc9b\" (UniqueName: \"kubernetes.io/projected/971ec54a-9d87-48ee-9465-b7f3d685822c-kube-api-access-5dc9b\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.312544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.312506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/971ec54a-9d87-48ee-9465-b7f3d685822c-tls-certs\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.312544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.312548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dc9b\" (UniqueName: \"kubernetes.io/projected/971ec54a-9d87-48ee-9465-b7f3d685822c-kube-api-access-5dc9b\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.312736 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.312574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/971ec54a-9d87-48ee-9465-b7f3d685822c-tmp\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.315042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.315010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/971ec54a-9d87-48ee-9465-b7f3d685822c-tmp\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.315195 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.315087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/971ec54a-9d87-48ee-9465-b7f3d685822c-tls-certs\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.326164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.326140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dc9b\" (UniqueName: \"kubernetes.io/projected/971ec54a-9d87-48ee-9465-b7f3d685822c-kube-api-access-5dc9b\") pod \"kube-auth-proxy-9b867fd64-rsmwb\" (UID: \"971ec54a-9d87-48ee-9465-b7f3d685822c\") " pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.394734 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.394659    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"
Sep 10 22:12:06.662917 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:06.662847    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb"]
Sep 10 22:12:06.665851 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:12:06.665817    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971ec54a_9d87_48ee_9465_b7f3d685822c.slice/crio-48e98e331094710876a76c0c1fcbf63c953d429138c5420415d9c149ce4f547f WatchSource:0}: Error finding container 48e98e331094710876a76c0c1fcbf63c953d429138c5420415d9c149ce4f547f: Status 404 returned error can't find the container with id 48e98e331094710876a76c0c1fcbf63c953d429138c5420415d9c149ce4f547f
Sep 10 22:12:07.327133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.327105    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"]
Sep 10 22:12:07.329751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.329730    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.332131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.331998    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 22:12:07.332131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.332011    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 22:12:07.332131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.332029    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 22:12:07.332364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.332254    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 22:12:07.332421 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.332376    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5548b\""
Sep 10 22:12:07.344174 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.344139    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"]
Sep 10 22:12:07.422482 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.422449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.422663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.422496    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f78r\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-kube-api-access-7f78r\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.422663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.422556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.422663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.422588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.422663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.422618    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.422663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.422658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/92c67686-c1bb-4998-87f9-69aba2d48f73-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.422869 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.422713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.523999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f78r\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-kube-api-access-7f78r\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.524046    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.524071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.524097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.524136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/92c67686-c1bb-4998-87f9-69aba2d48f73-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.524167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524427 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.524216    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.524819 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.524761    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.527226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.527201    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/92c67686-c1bb-4998-87f9-69aba2d48f73-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.527340 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.527258    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.527402 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.527363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.527495 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.527456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.532818 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.532776    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.533511 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.533473    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f78r\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-kube-api-access-7f78r\") pod \"istiod-openshift-gateway-7b7995d5cd-pd8v7\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.641028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.640954    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:07.656671 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.656639    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb" event={"ID":"971ec54a-9d87-48ee-9465-b7f3d685822c","Type":"ContainerStarted","Data":"48e98e331094710876a76c0c1fcbf63c953d429138c5420415d9c149ce4f547f"}
Sep 10 22:12:07.930805 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:07.930773    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"]
Sep 10 22:12:07.934351 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:12:07.934317    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c67686_c1bb_4998_87f9_69aba2d48f73.slice/crio-3bb73539c77fbfbd2d9af0cec7c006a17a7b1cddfa3ce8993ae9eee06c6722cd WatchSource:0}: Error finding container 3bb73539c77fbfbd2d9af0cec7c006a17a7b1cddfa3ce8993ae9eee06c6722cd: Status 404 returned error can't find the container with id 3bb73539c77fbfbd2d9af0cec7c006a17a7b1cddfa3ce8993ae9eee06c6722cd
Sep 10 22:12:08.662896 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:08.662859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7" event={"ID":"92c67686-c1bb-4998-87f9-69aba2d48f73","Type":"ContainerStarted","Data":"3bb73539c77fbfbd2d9af0cec7c006a17a7b1cddfa3ce8993ae9eee06c6722cd"}
Sep 10 22:12:10.679793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:10.679757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb" event={"ID":"971ec54a-9d87-48ee-9465-b7f3d685822c","Type":"ContainerStarted","Data":"c7259fe3e7c0275ab00824e28e6bb3b0c67d7599bba95920aea31cba9aff49e5"}
Sep 10 22:12:11.305814 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:11.305781    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:12:11.305885 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:11.305847    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:12:11.685467 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:11.685350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7" event={"ID":"92c67686-c1bb-4998-87f9-69aba2d48f73","Type":"ContainerStarted","Data":"254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b"}
Sep 10 22:12:11.685467 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:11.685417    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:11.709562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:11.709393    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9b867fd64-rsmwb" podStartSLOduration=2.804875764 podStartE2EDuration="5.709374972s" totalImagesPullingTime="2.904499208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:12:10.69952676 +0000 UTC m=+400.084032186" watchObservedRunningTime="2026-09-10 22:12:11.709374972 +0000 UTC m=+401.093880397"
Sep 10 22:12:11.709860 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:11.709807    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7" podStartSLOduration=1.3415547 podStartE2EDuration="4.709794804s" totalImagesPullingTime="3.368240104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:12:11.707907224 +0000 UTC m=+401.092412649" watchObservedRunningTime="2026-09-10 22:12:11.709794804 +0000 UTC m=+401.094300228"
Sep 10 22:12:12.587047 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.587018    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"]
Sep 10 22:12:12.589183 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.589162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.591598 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.591577    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ljcws\""
Sep 10 22:12:12.613568 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.613544    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"]
Sep 10 22:12:12.695023 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.694995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:12.769176 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769184    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769225    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/659c8c68-059b-41b0-ad38-1f4b441fd841-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769589 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769589 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mb9j\" (UniqueName: \"kubernetes.io/projected/659c8c68-059b-41b0-ad38-1f4b441fd841-kube-api-access-4mb9j\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.769589 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.769504    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/659c8c68-059b-41b0-ad38-1f4b441fd841-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870355    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mb9j\" (UniqueName: \"kubernetes.io/projected/659c8c68-059b-41b0-ad38-1f4b441fd841-kube-api-access-4mb9j\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870748 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870748 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.870930 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.870902    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.871172 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.871148    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.871250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.871200    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.871403 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.871378    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/659c8c68-059b-41b0-ad38-1f4b441fd841-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.872752 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.872735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.873062 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.873043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.880961 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.880930    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/659c8c68-059b-41b0-ad38-1f4b441fd841-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.881058 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.880980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mb9j\" (UniqueName: \"kubernetes.io/projected/659c8c68-059b-41b0-ad38-1f4b441fd841-kube-api-access-4mb9j\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj\" (UID: \"659c8c68-059b-41b0-ad38-1f4b441fd841\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:12.898910 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:12.898887    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:13.176311 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:13.176289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"]
Sep 10 22:12:13.178072 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:12:13.178046    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659c8c68_059b_41b0_ad38_1f4b441fd841.slice/crio-b6cfaea862ee1075aa7045ac171c84509f877267b9b9dc2cd7024b89bfe41611 WatchSource:0}: Error finding container b6cfaea862ee1075aa7045ac171c84509f877267b9b9dc2cd7024b89bfe41611: Status 404 returned error can't find the container with id b6cfaea862ee1075aa7045ac171c84509f877267b9b9dc2cd7024b89bfe41611
Sep 10 22:12:13.698008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:13.697967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj" event={"ID":"659c8c68-059b-41b0-ad38-1f4b441fd841","Type":"ContainerStarted","Data":"b6cfaea862ee1075aa7045ac171c84509f877267b9b9dc2cd7024b89bfe41611"}
Sep 10 22:12:15.890460 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:15.890400    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:12:15.890743 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:15.890500    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:12:15.890743 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:15.890533    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:12:16.713704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:16.713665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj" event={"ID":"659c8c68-059b-41b0-ad38-1f4b441fd841","Type":"ContainerStarted","Data":"090d68a97daa33e4cfcae1d9e47f37eae5b5cbc1eb826365c3ef619365417845"}
Sep 10 22:12:16.732875 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:16.732830    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj" podStartSLOduration=2.024978423 podStartE2EDuration="4.73281315s" totalImagesPullingTime="2.707834727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:12:16.731389086 +0000 UTC m=+406.115894511" watchObservedRunningTime="2026-09-10 22:12:16.73281315 +0000 UTC m=+406.117318574"
Sep 10 22:12:16.899515 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:16.899472    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:16.904943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:16.904882    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:17.718079 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:17.718054    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:17.719009 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:17.718991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj"
Sep 10 22:12:26.624960 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.624922    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"]
Sep 10 22:12:26.627515 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.627494    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:26.629920 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.629898    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xdjn7\""
Sep 10 22:12:26.630678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.630658    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 22:12:26.643628 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.643600    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"]
Sep 10 22:12:26.677113 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.677083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfb1b1da-5571-4944-a033-8a40166d518a-cert\") pod \"kserve-controller-manager-d6f4b65b8-b7xh5\" (UID: \"dfb1b1da-5571-4944-a033-8a40166d518a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:26.677252 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.677127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbc59\" (UniqueName: \"kubernetes.io/projected/dfb1b1da-5571-4944-a033-8a40166d518a-kube-api-access-xbc59\") pod \"kserve-controller-manager-d6f4b65b8-b7xh5\" (UID: \"dfb1b1da-5571-4944-a033-8a40166d518a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:26.778104 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.778071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfb1b1da-5571-4944-a033-8a40166d518a-cert\") pod \"kserve-controller-manager-d6f4b65b8-b7xh5\" (UID: \"dfb1b1da-5571-4944-a033-8a40166d518a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:26.778104 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.778106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbc59\" (UniqueName: \"kubernetes.io/projected/dfb1b1da-5571-4944-a033-8a40166d518a-kube-api-access-xbc59\") pod \"kserve-controller-manager-d6f4b65b8-b7xh5\" (UID: \"dfb1b1da-5571-4944-a033-8a40166d518a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:26.780673 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.780649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfb1b1da-5571-4944-a033-8a40166d518a-cert\") pod \"kserve-controller-manager-d6f4b65b8-b7xh5\" (UID: \"dfb1b1da-5571-4944-a033-8a40166d518a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:26.795605 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.795575    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbc59\" (UniqueName: \"kubernetes.io/projected/dfb1b1da-5571-4944-a033-8a40166d518a-kube-api-access-xbc59\") pod \"kserve-controller-manager-d6f4b65b8-b7xh5\" (UID: \"dfb1b1da-5571-4944-a033-8a40166d518a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:26.942116 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:26.942008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:27.218600 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:27.218575    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"]
Sep 10 22:12:27.221115 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:12:27.221080    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfb1b1da_5571_4944_a033_8a40166d518a.slice/crio-0e50d9f19d986afafb0cf7d8af74f6ecd85a566b7bcc04ebf44aec5acbb3d39f WatchSource:0}: Error finding container 0e50d9f19d986afafb0cf7d8af74f6ecd85a566b7bcc04ebf44aec5acbb3d39f: Status 404 returned error can't find the container with id 0e50d9f19d986afafb0cf7d8af74f6ecd85a566b7bcc04ebf44aec5acbb3d39f
Sep 10 22:12:27.757792 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:27.757761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5" event={"ID":"dfb1b1da-5571-4944-a033-8a40166d518a","Type":"ContainerStarted","Data":"0e50d9f19d986afafb0cf7d8af74f6ecd85a566b7bcc04ebf44aec5acbb3d39f"}
Sep 10 22:12:28.689984 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:28.689947    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"]
Sep 10 22:12:28.690251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:28.690194    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7" podUID="92c67686-c1bb-4998-87f9-69aba2d48f73" containerName="discovery" containerID="cri-o://254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b" gracePeriod=30
Sep 10 22:12:29.381243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.381220    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:29.401261 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.401243    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/92c67686-c1bb-4998-87f9-69aba2d48f73-local-certs\") pod \"92c67686-c1bb-4998-87f9-69aba2d48f73\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") "
Sep 10 22:12:29.401364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.401289    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-token\") pod \"92c67686-c1bb-4998-87f9-69aba2d48f73\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") "
Sep 10 22:12:29.401364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.401327    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-kubeconfig\") pod \"92c67686-c1bb-4998-87f9-69aba2d48f73\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") "
Sep 10 22:12:29.401364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.401352    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f78r\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-kube-api-access-7f78r\") pod \"92c67686-c1bb-4998-87f9-69aba2d48f73\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") "
Sep 10 22:12:29.401550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.401376    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-dns-cert\") pod \"92c67686-c1bb-4998-87f9-69aba2d48f73\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") "
Sep 10 22:12:29.401550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.401459    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-cacerts\") pod \"92c67686-c1bb-4998-87f9-69aba2d48f73\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") "
Sep 10 22:12:29.401550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.401500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-ca-configmap\") pod \"92c67686-c1bb-4998-87f9-69aba2d48f73\" (UID: \"92c67686-c1bb-4998-87f9-69aba2d48f73\") "
Sep 10 22:12:29.402510 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.402262    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "92c67686-c1bb-4998-87f9-69aba2d48f73" (UID: "92c67686-c1bb-4998-87f9-69aba2d48f73"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:12:29.404788 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.404757    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-kube-api-access-7f78r" (OuterVolumeSpecName: "kube-api-access-7f78r") pod "92c67686-c1bb-4998-87f9-69aba2d48f73" (UID: "92c67686-c1bb-4998-87f9-69aba2d48f73"). InnerVolumeSpecName "kube-api-access-7f78r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:12:29.404889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.404840    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92c67686-c1bb-4998-87f9-69aba2d48f73-local-certs" (OuterVolumeSpecName: "local-certs") pod "92c67686-c1bb-4998-87f9-69aba2d48f73" (UID: "92c67686-c1bb-4998-87f9-69aba2d48f73"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:12:29.404951 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.404917    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "92c67686-c1bb-4998-87f9-69aba2d48f73" (UID: "92c67686-c1bb-4998-87f9-69aba2d48f73"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:12:29.405526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.405492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "92c67686-c1bb-4998-87f9-69aba2d48f73" (UID: "92c67686-c1bb-4998-87f9-69aba2d48f73"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:12:29.405687 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.405662    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-cacerts" (OuterVolumeSpecName: "cacerts") pod "92c67686-c1bb-4998-87f9-69aba2d48f73" (UID: "92c67686-c1bb-4998-87f9-69aba2d48f73"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:12:29.405936 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.405910    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-token" (OuterVolumeSpecName: "istio-token") pod "92c67686-c1bb-4998-87f9-69aba2d48f73" (UID: "92c67686-c1bb-4998-87f9-69aba2d48f73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:12:29.502194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.502167    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-ca-configmap\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:12:29.502194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.502191    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/92c67686-c1bb-4998-87f9-69aba2d48f73-local-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:12:29.502194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.502200    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:12:29.502419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.502208    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-kubeconfig\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:12:29.502419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.502216    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f78r\" (UniqueName: \"kubernetes.io/projected/92c67686-c1bb-4998-87f9-69aba2d48f73-kube-api-access-7f78r\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:12:29.502419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.502225    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-istio-csr-dns-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:12:29.502419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.502235    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/92c67686-c1bb-4998-87f9-69aba2d48f73-cacerts\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:12:29.769787 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.769756    2563 generic.go:356] "Generic (PLEG): container finished" podID="92c67686-c1bb-4998-87f9-69aba2d48f73" containerID="254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b" exitCode=0
Sep 10 22:12:29.769969 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.769813    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"
Sep 10 22:12:29.769969 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.769834    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7" event={"ID":"92c67686-c1bb-4998-87f9-69aba2d48f73","Type":"ContainerDied","Data":"254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b"}
Sep 10 22:12:29.769969 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.769883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7" event={"ID":"92c67686-c1bb-4998-87f9-69aba2d48f73","Type":"ContainerDied","Data":"3bb73539c77fbfbd2d9af0cec7c006a17a7b1cddfa3ce8993ae9eee06c6722cd"}
Sep 10 22:12:29.769969 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.769904    2563 scope.go:117] "RemoveContainer" containerID="254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b"
Sep 10 22:12:29.771560 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.771477    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5" event={"ID":"dfb1b1da-5571-4944-a033-8a40166d518a","Type":"ContainerStarted","Data":"b09637ced2dbff345a2d80e539dde58adc71ac3b8a11ade46f2947ca01ed2b54"}
Sep 10 22:12:29.771639 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.771601    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:12:29.779406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.779385    2563 scope.go:117] "RemoveContainer" containerID="254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b"
Sep 10 22:12:29.779726 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:12:29.779707    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b\": container with ID starting with 254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b not found: ID does not exist" containerID="254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b"
Sep 10 22:12:29.779771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.779735    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b"} err="failed to get container status \"254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b\": rpc error: code = NotFound desc = could not find container \"254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b\": container with ID starting with 254e202285cdff2c6462ba77b2e3439be72fac85e6537bdb67d180de84794b5b not found: ID does not exist"
Sep 10 22:12:29.792162 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.792118    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5" podStartSLOduration=1.620897934 podStartE2EDuration="3.792102458s" totalImagesPullingTime="2.171204524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:12:29.790743243 +0000 UTC m=+419.175248668" watchObservedRunningTime="2026-09-10 22:12:29.792102458 +0000 UTC m=+419.176607884"
Sep 10 22:12:29.811612 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.811586    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"]
Sep 10 22:12:29.815923 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:29.815899    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-pd8v7"]
Sep 10 22:12:31.109215 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:12:31.109185    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c67686-c1bb-4998-87f9-69aba2d48f73" path="/var/lib/kubelet/pods/92c67686-c1bb-4998-87f9-69aba2d48f73/volumes"
Sep 10 22:13:00.782401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:00.782323    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b7xh5"
Sep 10 22:13:35.990109 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.990070    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"]
Sep 10 22:13:35.990611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.990341    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c67686-c1bb-4998-87f9-69aba2d48f73" containerName="discovery"
Sep 10 22:13:35.990611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.990353    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c67686-c1bb-4998-87f9-69aba2d48f73" containerName="discovery"
Sep 10 22:13:35.990611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.990408    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c67686-c1bb-4998-87f9-69aba2d48f73" containerName="discovery"
Sep 10 22:13:35.993090 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.993075    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:35.997716 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.997697    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d2hqd\""
Sep 10 22:13:35.997843 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.997832    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 22:13:35.998499 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:35.998480    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 22:13:36.007285 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.007259    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"]
Sep 10 22:13:36.016043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.016018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13bc7faa-75e7-437c-a8fc-d36eace89c0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:36.016159 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.016051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4whx\" (UniqueName: \"kubernetes.io/projected/13bc7faa-75e7-437c-a8fc-d36eace89c0e-kube-api-access-b4whx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:36.117481 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.117428    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13bc7faa-75e7-437c-a8fc-d36eace89c0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:36.117481 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.117483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4whx\" (UniqueName: \"kubernetes.io/projected/13bc7faa-75e7-437c-a8fc-d36eace89c0e-kube-api-access-b4whx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:36.117831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.117811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13bc7faa-75e7-437c-a8fc-d36eace89c0e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:36.126429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.126395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4whx\" (UniqueName: \"kubernetes.io/projected/13bc7faa-75e7-437c-a8fc-d36eace89c0e-kube-api-access-b4whx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:36.303042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.302958    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:36.578146 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:36.578121    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"]
Sep 10 22:13:36.580984 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:13:36.580959    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13bc7faa_75e7_437c_a8fc_d36eace89c0e.slice/crio-d1400788dbe3a85e9a6a8e73d62b2d41c4da1036c3d3a853c2b941ae58634450 WatchSource:0}: Error finding container d1400788dbe3a85e9a6a8e73d62b2d41c4da1036c3d3a853c2b941ae58634450: Status 404 returned error can't find the container with id d1400788dbe3a85e9a6a8e73d62b2d41c4da1036c3d3a853c2b941ae58634450
Sep 10 22:13:37.044930 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:37.044890    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm" event={"ID":"13bc7faa-75e7-437c-a8fc-d36eace89c0e","Type":"ContainerStarted","Data":"d1400788dbe3a85e9a6a8e73d62b2d41c4da1036c3d3a853c2b941ae58634450"}
Sep 10 22:13:38.020953 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.020908    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"]
Sep 10 22:13:38.024396 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.024366    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"
Sep 10 22:13:38.027520 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.027485    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 22:13:38.027799 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.027777    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hwp66\""
Sep 10 22:13:38.033174 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.033098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv4bk\" (UniqueName: \"kubernetes.io/projected/ad91562f-dc92-4593-908e-1e6119f2442c-kube-api-access-sv4bk\") pod \"dns-operator-controller-manager-7b84cd7c9c-l4tlz\" (UID: \"ad91562f-dc92-4593-908e-1e6119f2442c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"
Sep 10 22:13:38.036330 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.036260    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"]
Sep 10 22:13:38.134306 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.133922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv4bk\" (UniqueName: \"kubernetes.io/projected/ad91562f-dc92-4593-908e-1e6119f2442c-kube-api-access-sv4bk\") pod \"dns-operator-controller-manager-7b84cd7c9c-l4tlz\" (UID: \"ad91562f-dc92-4593-908e-1e6119f2442c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"
Sep 10 22:13:38.148302 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.148273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv4bk\" (UniqueName: \"kubernetes.io/projected/ad91562f-dc92-4593-908e-1e6119f2442c-kube-api-access-sv4bk\") pod \"dns-operator-controller-manager-7b84cd7c9c-l4tlz\" (UID: \"ad91562f-dc92-4593-908e-1e6119f2442c\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"
Sep 10 22:13:38.338701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:38.338611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"
Sep 10 22:13:39.197755 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:39.197722    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"]
Sep 10 22:13:39.198711 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:13:39.198682    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad91562f_dc92_4593_908e_1e6119f2442c.slice/crio-eae3dc4617dd8be606ecd8a47e21373d2d600266ade7d4677305a6be4eab04c4 WatchSource:0}: Error finding container eae3dc4617dd8be606ecd8a47e21373d2d600266ade7d4677305a6be4eab04c4: Status 404 returned error can't find the container with id eae3dc4617dd8be606ecd8a47e21373d2d600266ade7d4677305a6be4eab04c4
Sep 10 22:13:40.063164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.063130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz" event={"ID":"ad91562f-dc92-4593-908e-1e6119f2442c","Type":"ContainerStarted","Data":"eae3dc4617dd8be606ecd8a47e21373d2d600266ade7d4677305a6be4eab04c4"}
Sep 10 22:13:40.713281 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.713236    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-dqts2"]
Sep 10 22:13:40.716948 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.716922    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-dqts2"
Sep 10 22:13:40.719805 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.719779    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-6m9ks\""
Sep 10 22:13:40.728346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.728321    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-dqts2"]
Sep 10 22:13:40.755923 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.755888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld8bn\" (UniqueName: \"kubernetes.io/projected/2e862025-2635-45ab-b18e-f5cbab619dc1-kube-api-access-ld8bn\") pod \"authorino-operator-76b478bd78-dqts2\" (UID: \"2e862025-2635-45ab-b18e-f5cbab619dc1\") " pod="kuadrant-system/authorino-operator-76b478bd78-dqts2"
Sep 10 22:13:40.857194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.857114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld8bn\" (UniqueName: \"kubernetes.io/projected/2e862025-2635-45ab-b18e-f5cbab619dc1-kube-api-access-ld8bn\") pod \"authorino-operator-76b478bd78-dqts2\" (UID: \"2e862025-2635-45ab-b18e-f5cbab619dc1\") " pod="kuadrant-system/authorino-operator-76b478bd78-dqts2"
Sep 10 22:13:40.873654 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:40.873617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld8bn\" (UniqueName: \"kubernetes.io/projected/2e862025-2635-45ab-b18e-f5cbab619dc1-kube-api-access-ld8bn\") pod \"authorino-operator-76b478bd78-dqts2\" (UID: \"2e862025-2635-45ab-b18e-f5cbab619dc1\") " pod="kuadrant-system/authorino-operator-76b478bd78-dqts2"
Sep 10 22:13:41.030196 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:41.030155    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-dqts2"
Sep 10 22:13:41.075947 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:41.075852    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm" event={"ID":"13bc7faa-75e7-437c-a8fc-d36eace89c0e","Type":"ContainerStarted","Data":"d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692"}
Sep 10 22:13:41.076137 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:41.075984    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:41.099035 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:41.098976    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm" podStartSLOduration=2.103542003 podStartE2EDuration="6.098958603s" totalImagesPullingTime="3.9954166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:13:41.096803284 +0000 UTC m=+490.481308709" watchObservedRunningTime="2026-09-10 22:13:41.098958603 +0000 UTC m=+490.483464028"
Sep 10 22:13:41.318966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:41.318921    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-dqts2"]
Sep 10 22:13:41.477154 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:13:41.477123    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e862025_2635_45ab_b18e_f5cbab619dc1.slice/crio-78b8889fe429c0cd2d3150e882679ba4911f6c4c1fd019d84e5d5d29167a477e WatchSource:0}: Error finding container 78b8889fe429c0cd2d3150e882679ba4911f6c4c1fd019d84e5d5d29167a477e: Status 404 returned error can't find the container with id 78b8889fe429c0cd2d3150e882679ba4911f6c4c1fd019d84e5d5d29167a477e
Sep 10 22:13:42.081687 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:42.081654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz" event={"ID":"ad91562f-dc92-4593-908e-1e6119f2442c","Type":"ContainerStarted","Data":"02161fc584936e30ea54b6cfca9ddd757755e3f262086c6cb8c2f2c9bd88dcd6"}
Sep 10 22:13:42.082161 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:42.081713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"
Sep 10 22:13:42.082948 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:42.082922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-dqts2" event={"ID":"2e862025-2635-45ab-b18e-f5cbab619dc1","Type":"ContainerStarted","Data":"78b8889fe429c0cd2d3150e882679ba4911f6c4c1fd019d84e5d5d29167a477e"}
Sep 10 22:13:42.099057 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:42.099000    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz" podStartSLOduration=2.765637911 podStartE2EDuration="5.098980208s" totalImagesPullingTime="2.333342297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:13:42.098387107 +0000 UTC m=+491.482892523" watchObservedRunningTime="2026-09-10 22:13:42.098980208 +0000 UTC m=+491.483485635"
Sep 10 22:13:43.005641 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.005601    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"]
Sep 10 22:13:43.008946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.008923    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:43.011162 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.011136    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-w4rj7\""
Sep 10 22:13:43.021100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.021073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"]
Sep 10 22:13:43.074667 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.074637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jgjv\" (UniqueName: \"kubernetes.io/projected/53c64553-b34f-4aeb-93e0-56c9b631b366-kube-api-access-9jgjv\") pod \"limitador-operator-controller-manager-74495c69f8-d6l8b\" (UID: \"53c64553-b34f-4aeb-93e0-56c9b631b366\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:43.176063 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.176020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jgjv\" (UniqueName: \"kubernetes.io/projected/53c64553-b34f-4aeb-93e0-56c9b631b366-kube-api-access-9jgjv\") pod \"limitador-operator-controller-manager-74495c69f8-d6l8b\" (UID: \"53c64553-b34f-4aeb-93e0-56c9b631b366\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:43.184469 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.184429    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jgjv\" (UniqueName: \"kubernetes.io/projected/53c64553-b34f-4aeb-93e0-56c9b631b366-kube-api-access-9jgjv\") pod \"limitador-operator-controller-manager-74495c69f8-d6l8b\" (UID: \"53c64553-b34f-4aeb-93e0-56c9b631b366\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:43.322689 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.322588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:43.517849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.517820    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"]
Sep 10 22:13:43.521251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.521232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.523490 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.523468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vblj2\""
Sep 10 22:13:43.523844 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.523823    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 22:13:43.524050 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.524034    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 22:13:43.530249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.530164    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"]
Sep 10 22:13:43.579484 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.579423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e813661-5255-4c9f-b12e-29c56d887527-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.579622 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.579514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e813661-5255-4c9f-b12e-29c56d887527-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.579622 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.579543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvf22\" (UniqueName: \"kubernetes.io/projected/0e813661-5255-4c9f-b12e-29c56d887527-kube-api-access-cvf22\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.680253 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.680217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e813661-5255-4c9f-b12e-29c56d887527-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.680407 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.680261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e813661-5255-4c9f-b12e-29c56d887527-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.680407 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.680295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvf22\" (UniqueName: \"kubernetes.io/projected/0e813661-5255-4c9f-b12e-29c56d887527-kube-api-access-cvf22\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.681583 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.681552    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e813661-5255-4c9f-b12e-29c56d887527-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.683113 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.683090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e813661-5255-4c9f-b12e-29c56d887527-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.688149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.688127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvf22\" (UniqueName: \"kubernetes.io/projected/0e813661-5255-4c9f-b12e-29c56d887527-kube-api-access-cvf22\") pod \"kuadrant-console-plugin-fbdd4f445-s7kqv\" (UID: \"0e813661-5255-4c9f-b12e-29c56d887527\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:43.721100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.721076    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"]
Sep 10 22:13:43.723555 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:13:43.723526    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c64553_b34f_4aeb_93e0_56c9b631b366.slice/crio-54fba7091013656558cd7f128c141d0cbb67fff068dff41d191b1e22e6ae08e3 WatchSource:0}: Error finding container 54fba7091013656558cd7f128c141d0cbb67fff068dff41d191b1e22e6ae08e3: Status 404 returned error can't find the container with id 54fba7091013656558cd7f128c141d0cbb67fff068dff41d191b1e22e6ae08e3
Sep 10 22:13:43.843281 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:43.843195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"
Sep 10 22:13:44.094538 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:44.094427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-dqts2" event={"ID":"2e862025-2635-45ab-b18e-f5cbab619dc1","Type":"ContainerStarted","Data":"83602a5db9587dd07dd03469d71aed2c808d2e6917d603d2b09c0a9d592f5362"}
Sep 10 22:13:44.094708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:44.094556    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-dqts2"
Sep 10 22:13:44.095928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:44.095905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b" event={"ID":"53c64553-b34f-4aeb-93e0-56c9b631b366","Type":"ContainerStarted","Data":"54fba7091013656558cd7f128c141d0cbb67fff068dff41d191b1e22e6ae08e3"}
Sep 10 22:13:44.113160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:44.113102    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-dqts2" podStartSLOduration=2.078341987 podStartE2EDuration="4.113081193s" totalImagesPullingTime="2.034739206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:13:44.112034082 +0000 UTC m=+493.496539507" watchObservedRunningTime="2026-09-10 22:13:44.113081193 +0000 UTC m=+493.497586620"
Sep 10 22:13:44.120848 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:44.120825    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv"]
Sep 10 22:13:45.103168 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:45.103126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv" event={"ID":"0e813661-5255-4c9f-b12e-29c56d887527","Type":"ContainerStarted","Data":"5dc0effc3d4a019bd30f7841eff02177c48de7b18b2e04d55b8225941b7cd849"}
Sep 10 22:13:46.110373 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:46.110329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b" event={"ID":"53c64553-b34f-4aeb-93e0-56c9b631b366","Type":"ContainerStarted","Data":"5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b"}
Sep 10 22:13:46.110804 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:46.110466    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:46.150734 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:46.150616    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b" podStartSLOduration=2.009392359 podStartE2EDuration="4.150597247s" totalImagesPullingTime="2.141204888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:13:46.148563118 +0000 UTC m=+495.533068549" watchObservedRunningTime="2026-09-10 22:13:46.150597247 +0000 UTC m=+495.535102671"
Sep 10 22:13:50.131038 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:50.130999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv" event={"ID":"0e813661-5255-4c9f-b12e-29c56d887527","Type":"ContainerStarted","Data":"2f79b3fe5feceba45338362bd310d0c31cffbd1af08191a7a4895ca64e72cc30"}
Sep 10 22:13:50.145606 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:50.145558    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-s7kqv" podStartSLOduration=2.103473162 podStartE2EDuration="7.145542557s" totalImagesPullingTime="5.042069395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:13:50.144906413 +0000 UTC m=+499.529411837" watchObservedRunningTime="2026-09-10 22:13:50.145542557 +0000 UTC m=+499.530048018"
Sep 10 22:13:52.086101 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:52.086070    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:53.090939 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:53.090908    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-l4tlz"
Sep 10 22:13:54.192410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.192381    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"]
Sep 10 22:13:54.192835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.192582    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" containerName="manager" containerID="cri-o://d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692" gracePeriod=2
Sep 10 22:13:54.195094 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.195062    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.196063 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.196038    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"]
Sep 10 22:13:54.222299 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.222252    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"]
Sep 10 22:13:54.222558 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.222530    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" containerName="manager" containerID="cri-o://5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b" gracePeriod=2
Sep 10 22:13:54.224585 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.224561    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"]
Sep 10 22:13:54.225114 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.225093    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" containerName="manager"
Sep 10 22:13:54.225203 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.225144    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" containerName="manager"
Sep 10 22:13:54.225260 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.225248    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" containerName="manager"
Sep 10 22:13:54.225305 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.225242    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.226959 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.226912    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-d6l8b\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.228322 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.228300    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.228485 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.228468    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"]
Sep 10 22:13:54.228622 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.228391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.230229 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.230202    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.232041 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.232010    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-d6l8b\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.244991 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.244951    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"]
Sep 10 22:13:54.248287 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.248264    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"]
Sep 10 22:13:54.248579 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.248566    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" containerName="manager"
Sep 10 22:13:54.248632 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.248581    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" containerName="manager"
Sep 10 22:13:54.248664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.248643    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" containerName="manager"
Sep 10 22:13:54.251791 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.251777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"
Sep 10 22:13:54.260908 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.260875    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-d6l8b\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.262747 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.262707    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.263380 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.263357    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"]
Sep 10 22:13:54.280431 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.280401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/898c3304-5f94-4b2f-acea-2fe64d3c77ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9ft48\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.280575 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.280468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlv48\" (UniqueName: \"kubernetes.io/projected/898c3304-5f94-4b2f-acea-2fe64d3c77ec-kube-api-access-dlv48\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9ft48\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.286342 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.286311    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-d6l8b\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.288236 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.288210    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.381240 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.381205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/898c3304-5f94-4b2f-acea-2fe64d3c77ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9ft48\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.381403 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.381254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlv48\" (UniqueName: \"kubernetes.io/projected/898c3304-5f94-4b2f-acea-2fe64d3c77ec-kube-api-access-dlv48\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9ft48\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.381403 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.381321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv9d6\" (UniqueName: \"kubernetes.io/projected/1633881a-0359-42f7-ab23-37d1dbaa9fea-kube-api-access-fv9d6\") pod \"limitador-operator-controller-manager-74495c69f8-hx57f\" (UID: \"1633881a-0359-42f7-ab23-37d1dbaa9fea\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"
Sep 10 22:13:54.381598 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.381575    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/898c3304-5f94-4b2f-acea-2fe64d3c77ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9ft48\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.389887 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.389851    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlv48\" (UniqueName: \"kubernetes.io/projected/898c3304-5f94-4b2f-acea-2fe64d3c77ec-kube-api-access-dlv48\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9ft48\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.459351 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.459330    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.461483 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.461454    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-d6l8b\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.461775 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.461762    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.463164 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.463145    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.464876 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.464858    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-d6l8b\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:54.466608 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:54.466591    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-4hcxm\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:54.481970 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.481945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv9d6\" (UniqueName: \"kubernetes.io/projected/1633881a-0359-42f7-ab23-37d1dbaa9fea-kube-api-access-fv9d6\") pod \"limitador-operator-controller-manager-74495c69f8-hx57f\" (UID: \"1633881a-0359-42f7-ab23-37d1dbaa9fea\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"
Sep 10 22:13:54.490392 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.490371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv9d6\" (UniqueName: \"kubernetes.io/projected/1633881a-0359-42f7-ab23-37d1dbaa9fea-kube-api-access-fv9d6\") pod \"limitador-operator-controller-manager-74495c69f8-hx57f\" (UID: \"1633881a-0359-42f7-ab23-37d1dbaa9fea\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"
Sep 10 22:13:54.582260 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.582227    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4whx\" (UniqueName: \"kubernetes.io/projected/13bc7faa-75e7-437c-a8fc-d36eace89c0e-kube-api-access-b4whx\") pod \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") "
Sep 10 22:13:54.582426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.582360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13bc7faa-75e7-437c-a8fc-d36eace89c0e-extensions-socket-volume\") pod \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\" (UID: \"13bc7faa-75e7-437c-a8fc-d36eace89c0e\") "
Sep 10 22:13:54.582426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.582388    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jgjv\" (UniqueName: \"kubernetes.io/projected/53c64553-b34f-4aeb-93e0-56c9b631b366-kube-api-access-9jgjv\") pod \"53c64553-b34f-4aeb-93e0-56c9b631b366\" (UID: \"53c64553-b34f-4aeb-93e0-56c9b631b366\") "
Sep 10 22:13:54.582876 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.582843    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13bc7faa-75e7-437c-a8fc-d36eace89c0e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "13bc7faa-75e7-437c-a8fc-d36eace89c0e" (UID: "13bc7faa-75e7-437c-a8fc-d36eace89c0e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:13:54.584698 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.584671    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13bc7faa-75e7-437c-a8fc-d36eace89c0e-kube-api-access-b4whx" (OuterVolumeSpecName: "kube-api-access-b4whx") pod "13bc7faa-75e7-437c-a8fc-d36eace89c0e" (UID: "13bc7faa-75e7-437c-a8fc-d36eace89c0e"). InnerVolumeSpecName "kube-api-access-b4whx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:13:54.584698 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.584675    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c64553-b34f-4aeb-93e0-56c9b631b366-kube-api-access-9jgjv" (OuterVolumeSpecName: "kube-api-access-9jgjv") pod "53c64553-b34f-4aeb-93e0-56c9b631b366" (UID: "53c64553-b34f-4aeb-93e0-56c9b631b366"). InnerVolumeSpecName "kube-api-access-9jgjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:13:54.600200 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.600176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:54.607304 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.607283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"
Sep 10 22:13:54.683981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.683924    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/13bc7faa-75e7-437c-a8fc-d36eace89c0e-extensions-socket-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:13:54.683981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.683952    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jgjv\" (UniqueName: \"kubernetes.io/projected/53c64553-b34f-4aeb-93e0-56c9b631b366-kube-api-access-9jgjv\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:13:54.683981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.683961    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4whx\" (UniqueName: \"kubernetes.io/projected/13bc7faa-75e7-437c-a8fc-d36eace89c0e-kube-api-access-b4whx\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:13:54.881116 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.881091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"]
Sep 10 22:13:54.883366 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:13:54.883342    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1633881a_0359_42f7_ab23_37d1dbaa9fea.slice/crio-c7eb90e5efe7cd4820db533d5579a925b46b97e6d69832901e85e26e8d1adcad WatchSource:0}: Error finding container c7eb90e5efe7cd4820db533d5579a925b46b97e6d69832901e85e26e8d1adcad: Status 404 returned error can't find the container with id c7eb90e5efe7cd4820db533d5579a925b46b97e6d69832901e85e26e8d1adcad
Sep 10 22:13:54.889350 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:54.889326    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"]
Sep 10 22:13:54.892272 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:13:54.892239    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898c3304_5f94_4b2f_acea_2fe64d3c77ec.slice/crio-1b477d47e028f56207f99e883d9f9ed83e0ab113f720de8976945d415d61424c WatchSource:0}: Error finding container 1b477d47e028f56207f99e883d9f9ed83e0ab113f720de8976945d415d61424c: Status 404 returned error can't find the container with id 1b477d47e028f56207f99e883d9f9ed83e0ab113f720de8976945d415d61424c
Sep 10 22:13:55.109642 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.109612    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" path="/var/lib/kubelet/pods/13bc7faa-75e7-437c-a8fc-d36eace89c0e/volumes"
Sep 10 22:13:55.109925 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.109913    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53c64553-b34f-4aeb-93e0-56c9b631b366" path="/var/lib/kubelet/pods/53c64553-b34f-4aeb-93e0-56c9b631b366/volumes"
Sep 10 22:13:55.110249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.110236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-dqts2"
Sep 10 22:13:55.155897 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.155806    2563 generic.go:356] "Generic (PLEG): container finished" podID="53c64553-b34f-4aeb-93e0-56c9b631b366" containerID="5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b" exitCode=0
Sep 10 22:13:55.155897 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.155885    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-d6l8b"
Sep 10 22:13:55.156094 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.155903    2563 scope.go:117] "RemoveContainer" containerID="5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b"
Sep 10 22:13:55.158078 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.158045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48" event={"ID":"898c3304-5f94-4b2f-acea-2fe64d3c77ec","Type":"ContainerStarted","Data":"54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36"}
Sep 10 22:13:55.158213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.158156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48" event={"ID":"898c3304-5f94-4b2f-acea-2fe64d3c77ec","Type":"ContainerStarted","Data":"1b477d47e028f56207f99e883d9f9ed83e0ab113f720de8976945d415d61424c"}
Sep 10 22:13:55.158213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.158201    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:13:55.160073 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.160053    2563 generic.go:356] "Generic (PLEG): container finished" podID="13bc7faa-75e7-437c-a8fc-d36eace89c0e" containerID="d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692" exitCode=0
Sep 10 22:13:55.160202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.160125    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-4hcxm"
Sep 10 22:13:55.162164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.162139    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f" event={"ID":"1633881a-0359-42f7-ab23-37d1dbaa9fea","Type":"ContainerStarted","Data":"b73fa434c255d25a74b9e255145ac1da964a80766c81230ce05854332bf87f33"}
Sep 10 22:13:55.162305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.162210    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f" event={"ID":"1633881a-0359-42f7-ab23-37d1dbaa9fea","Type":"ContainerStarted","Data":"c7eb90e5efe7cd4820db533d5579a925b46b97e6d69832901e85e26e8d1adcad"}
Sep 10 22:13:55.162305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.162256    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"
Sep 10 22:13:55.165955 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.165935    2563 scope.go:117] "RemoveContainer" containerID="5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b"
Sep 10 22:13:55.166231 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:55.166207    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b\": container with ID starting with 5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b not found: ID does not exist" containerID="5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b"
Sep 10 22:13:55.166274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.166244    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b"} err="failed to get container status \"5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b\": rpc error: code = NotFound desc = could not find container \"5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b\": container with ID starting with 5380f09a8534ac27705493c021d8b1bd57c2ee29eb36fb4e8a2ebee7778ad22b not found: ID does not exist"
Sep 10 22:13:55.166274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.166267    2563 scope.go:117] "RemoveContainer" containerID="d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692"
Sep 10 22:13:55.173653 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.173636    2563 scope.go:117] "RemoveContainer" containerID="d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692"
Sep 10 22:13:55.173922 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:13:55.173899    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692\": container with ID starting with d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692 not found: ID does not exist" containerID="d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692"
Sep 10 22:13:55.174027 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.173925    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692"} err="failed to get container status \"d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692\": rpc error: code = NotFound desc = could not find container \"d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692\": container with ID starting with d82ea3dab7bc7a630ec0d024140880e711104ae62bfa70b947a7611e930bb692 not found: ID does not exist"
Sep 10 22:13:55.178753 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.178715    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48" podStartSLOduration=1.178702002 podStartE2EDuration="1.178702002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:13:55.178456038 +0000 UTC m=+504.562961454" watchObservedRunningTime="2026-09-10 22:13:55.178702002 +0000 UTC m=+504.563207428"
Sep 10 22:13:55.194203 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:13:55.194160    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f" podStartSLOduration=1.194145754 podStartE2EDuration="1.194145754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:13:55.193411125 +0000 UTC m=+504.577916672" watchObservedRunningTime="2026-09-10 22:13:55.194145754 +0000 UTC m=+504.578651178"
Sep 10 22:14:06.171828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:06.171796    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:14:06.172233 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:06.171882    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-hx57f"
Sep 10 22:14:11.521833 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:11.521797    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"]
Sep 10 22:14:11.522336 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:11.521994    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48" podUID="898c3304-5f94-4b2f-acea-2fe64d3c77ec" containerName="manager" containerID="cri-o://54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36" gracePeriod=10
Sep 10 22:14:11.968904 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:11.968883    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:14:12.127293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.127215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlv48\" (UniqueName: \"kubernetes.io/projected/898c3304-5f94-4b2f-acea-2fe64d3c77ec-kube-api-access-dlv48\") pod \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") "
Sep 10 22:14:12.127293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.127284    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/898c3304-5f94-4b2f-acea-2fe64d3c77ec-extensions-socket-volume\") pod \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\" (UID: \"898c3304-5f94-4b2f-acea-2fe64d3c77ec\") "
Sep 10 22:14:12.127719 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.127692    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/898c3304-5f94-4b2f-acea-2fe64d3c77ec-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "898c3304-5f94-4b2f-acea-2fe64d3c77ec" (UID: "898c3304-5f94-4b2f-acea-2fe64d3c77ec"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:14:12.129533 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.129513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/898c3304-5f94-4b2f-acea-2fe64d3c77ec-kube-api-access-dlv48" (OuterVolumeSpecName: "kube-api-access-dlv48") pod "898c3304-5f94-4b2f-acea-2fe64d3c77ec" (UID: "898c3304-5f94-4b2f-acea-2fe64d3c77ec"). InnerVolumeSpecName "kube-api-access-dlv48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:14:12.228428 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.228397    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlv48\" (UniqueName: \"kubernetes.io/projected/898c3304-5f94-4b2f-acea-2fe64d3c77ec-kube-api-access-dlv48\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:14:12.228428 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.228423    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/898c3304-5f94-4b2f-acea-2fe64d3c77ec-extensions-socket-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:14:12.243396 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.243371    2563 generic.go:356] "Generic (PLEG): container finished" podID="898c3304-5f94-4b2f-acea-2fe64d3c77ec" containerID="54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36" exitCode=0
Sep 10 22:14:12.243538 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.243413    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"
Sep 10 22:14:12.243538 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.243461    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48" event={"ID":"898c3304-5f94-4b2f-acea-2fe64d3c77ec","Type":"ContainerDied","Data":"54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36"}
Sep 10 22:14:12.243538 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.243503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48" event={"ID":"898c3304-5f94-4b2f-acea-2fe64d3c77ec","Type":"ContainerDied","Data":"1b477d47e028f56207f99e883d9f9ed83e0ab113f720de8976945d415d61424c"}
Sep 10 22:14:12.243538 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.243524    2563 scope.go:117] "RemoveContainer" containerID="54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36"
Sep 10 22:14:12.251616 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.251596    2563 scope.go:117] "RemoveContainer" containerID="54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36"
Sep 10 22:14:12.251877 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:14:12.251857    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36\": container with ID starting with 54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36 not found: ID does not exist" containerID="54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36"
Sep 10 22:14:12.251919 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.251885    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36"} err="failed to get container status \"54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36\": rpc error: code = NotFound desc = could not find container \"54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36\": container with ID starting with 54a59f92a20725b66fe16fce61502323c64687cde961ead2b627c32dac922e36 not found: ID does not exist"
Sep 10 22:14:12.262833 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.262813    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"]
Sep 10 22:14:12.269177 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:12.269156    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9ft48"]
Sep 10 22:14:13.110227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:13.110194    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="898c3304-5f94-4b2f-acea-2fe64d3c77ec" path="/var/lib/kubelet/pods/898c3304-5f94-4b2f-acea-2fe64d3c77ec/volumes"
Sep 10 22:14:27.781119 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.781043    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"]
Sep 10 22:14:27.781536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.781470    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="898c3304-5f94-4b2f-acea-2fe64d3c77ec" containerName="manager"
Sep 10 22:14:27.781536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.781485    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="898c3304-5f94-4b2f-acea-2fe64d3c77ec" containerName="manager"
Sep 10 22:14:27.781675 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.781576    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="898c3304-5f94-4b2f-acea-2fe64d3c77ec" containerName="manager"
Sep 10 22:14:27.783561 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.783538    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.785935 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.785914    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-lp95x\""
Sep 10 22:14:27.795909 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.795885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"]
Sep 10 22:14:27.854846 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.854820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.854981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.854862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9pfb\" (UniqueName: \"kubernetes.io/projected/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-kube-api-access-z9pfb\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.854981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.854886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.854981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.854943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.854981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.854969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.855142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.854988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.855142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.855029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.855142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.855057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.855142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.855074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956381    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9pfb\" (UniqueName: \"kubernetes.io/projected/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-kube-api-access-z9pfb\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.956972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.957077 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.956983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.957194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.957173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.957372 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.957351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.958858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.958839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.959119 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.959102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.964060 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.964035    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9pfb\" (UniqueName: \"kubernetes.io/projected/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-kube-api-access-z9pfb\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:27.964060 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:27.964047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-2784q\" (UID: \"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:28.094414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:28.094327    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:28.366910 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:28.366885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"]
Sep 10 22:14:28.368855 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:14:28.368833    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c8f1cb2_bf12_48a1_a2ab_dcca5b0d53ec.slice/crio-ba382455eaa006ed7afbe8f552ea8d3e54893acadf383582df621159d024b1f8 WatchSource:0}: Error finding container ba382455eaa006ed7afbe8f552ea8d3e54893acadf383582df621159d024b1f8: Status 404 returned error can't find the container with id ba382455eaa006ed7afbe8f552ea8d3e54893acadf383582df621159d024b1f8
Sep 10 22:14:28.371042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:28.371012    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:14:28.371106 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:28.371077    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:14:28.371144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:28.371106    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:14:29.317973 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:29.317940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" event={"ID":"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec","Type":"ContainerStarted","Data":"30e7232d27128a30601eff503ee6c249c5a76cfe47a874f5c0d44453eaa8c930"}
Sep 10 22:14:29.318391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:29.317987    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" event={"ID":"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec","Type":"ContainerStarted","Data":"ba382455eaa006ed7afbe8f552ea8d3e54893acadf383582df621159d024b1f8"}
Sep 10 22:14:29.336171 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:29.336114    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" podStartSLOduration=2.336099701 podStartE2EDuration="2.336099701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:29.33525326 +0000 UTC m=+538.719758687" watchObservedRunningTime="2026-09-10 22:14:29.336099701 +0000 UTC m=+538.720605125"
Sep 10 22:14:30.095620 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:30.095586    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:30.100654 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:30.100631    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:30.323159 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:30.323136    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:30.324669 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:30.324651    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:14:31.532004 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.531969    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-nwj8f"]
Sep 10 22:14:31.534949 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.534933    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.537354 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.537329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 22:14:31.544644 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.544621    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-nwj8f"]
Sep 10 22:14:31.583677 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.583647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t75mf\" (UniqueName: \"kubernetes.io/projected/f3012130-8285-40b0-9906-c2d2e8d8d6a8-kube-api-access-t75mf\") pod \"limitador-limitador-8588749977-nwj8f\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") " pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.583959 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.583929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3012130-8285-40b0-9906-c2d2e8d8d6a8-config-file\") pod \"limitador-limitador-8588749977-nwj8f\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") " pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.619013 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.618981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-nwj8f"]
Sep 10 22:14:31.684879 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.684841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t75mf\" (UniqueName: \"kubernetes.io/projected/f3012130-8285-40b0-9906-c2d2e8d8d6a8-kube-api-access-t75mf\") pod \"limitador-limitador-8588749977-nwj8f\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") " pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.685066 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.684923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3012130-8285-40b0-9906-c2d2e8d8d6a8-config-file\") pod \"limitador-limitador-8588749977-nwj8f\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") " pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.685573 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.685550    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3012130-8285-40b0-9906-c2d2e8d8d6a8-config-file\") pod \"limitador-limitador-8588749977-nwj8f\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") " pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.692450 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.692419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t75mf\" (UniqueName: \"kubernetes.io/projected/f3012130-8285-40b0-9906-c2d2e8d8d6a8-kube-api-access-t75mf\") pod \"limitador-limitador-8588749977-nwj8f\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") " pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.839202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.839124    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:14:31.841210 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.841195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:31.844957 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.844929    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:31.851220 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.851201    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:14:31.868206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.868179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:14:31.886832 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.886801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/502c096c-f9b7-4031-9f45-c8ab5f467fd0-config-file\") pod \"limitador-limitador-84994f7ffc-d7djs\" (UID: \"502c096c-f9b7-4031-9f45-c8ab5f467fd0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:31.886994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.886850    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xkf9\" (UniqueName: \"kubernetes.io/projected/502c096c-f9b7-4031-9f45-c8ab5f467fd0-kube-api-access-6xkf9\") pod \"limitador-limitador-84994f7ffc-d7djs\" (UID: \"502c096c-f9b7-4031-9f45-c8ab5f467fd0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:31.987604 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.987573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/502c096c-f9b7-4031-9f45-c8ab5f467fd0-config-file\") pod \"limitador-limitador-84994f7ffc-d7djs\" (UID: \"502c096c-f9b7-4031-9f45-c8ab5f467fd0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:31.987756 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.987613    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xkf9\" (UniqueName: \"kubernetes.io/projected/502c096c-f9b7-4031-9f45-c8ab5f467fd0-kube-api-access-6xkf9\") pod \"limitador-limitador-84994f7ffc-d7djs\" (UID: \"502c096c-f9b7-4031-9f45-c8ab5f467fd0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:31.988251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.988231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/502c096c-f9b7-4031-9f45-c8ab5f467fd0-config-file\") pod \"limitador-limitador-84994f7ffc-d7djs\" (UID: \"502c096c-f9b7-4031-9f45-c8ab5f467fd0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:31.995551 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:31.995527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xkf9\" (UniqueName: \"kubernetes.io/projected/502c096c-f9b7-4031-9f45-c8ab5f467fd0-kube-api-access-6xkf9\") pod \"limitador-limitador-84994f7ffc-d7djs\" (UID: \"502c096c-f9b7-4031-9f45-c8ab5f467fd0\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:32.116384 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.116361    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-nwj8f"]
Sep 10 22:14:32.118484 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:14:32.118429    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3012130_8285_40b0_9906_c2d2e8d8d6a8.slice/crio-3e6f22ed4a9cd9859e3b4ad3e35b964422a231a148bbdac632512ea5a844cc12 WatchSource:0}: Error finding container 3e6f22ed4a9cd9859e3b4ad3e35b964422a231a148bbdac632512ea5a844cc12: Status 404 returned error can't find the container with id 3e6f22ed4a9cd9859e3b4ad3e35b964422a231a148bbdac632512ea5a844cc12
Sep 10 22:14:32.153031 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.153003    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:32.334182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.334151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f" event={"ID":"f3012130-8285-40b0-9906-c2d2e8d8d6a8","Type":"ContainerStarted","Data":"3e6f22ed4a9cd9859e3b4ad3e35b964422a231a148bbdac632512ea5a844cc12"}
Sep 10 22:14:32.430595 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.430519    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:14:32.434096 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:14:32.434058    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502c096c_f9b7_4031_9f45_c8ab5f467fd0.slice/crio-da8343352536ce3e9969a42d50a2a0aecebd8eaf075ca756861af4a538f2e0b6 WatchSource:0}: Error finding container da8343352536ce3e9969a42d50a2a0aecebd8eaf075ca756861af4a538f2e0b6: Status 404 returned error can't find the container with id da8343352536ce3e9969a42d50a2a0aecebd8eaf075ca756861af4a538f2e0b6
Sep 10 22:14:32.597356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.597326    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-l974l"]
Sep 10 22:14:32.600112 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.600092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l974l"
Sep 10 22:14:32.602227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.602203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5kzlj\""
Sep 10 22:14:32.607710 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.607683    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l974l"]
Sep 10 22:14:32.693814 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.693739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gjpz\" (UniqueName: \"kubernetes.io/projected/fe5995eb-498a-40c7-9176-e6fac60a015d-kube-api-access-9gjpz\") pod \"authorino-59c974bd48-l974l\" (UID: \"fe5995eb-498a-40c7-9176-e6fac60a015d\") " pod="kuadrant-system/authorino-59c974bd48-l974l"
Sep 10 22:14:32.795042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.795003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gjpz\" (UniqueName: \"kubernetes.io/projected/fe5995eb-498a-40c7-9176-e6fac60a015d-kube-api-access-9gjpz\") pod \"authorino-59c974bd48-l974l\" (UID: \"fe5995eb-498a-40c7-9176-e6fac60a015d\") " pod="kuadrant-system/authorino-59c974bd48-l974l"
Sep 10 22:14:32.803116 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.803091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gjpz\" (UniqueName: \"kubernetes.io/projected/fe5995eb-498a-40c7-9176-e6fac60a015d-kube-api-access-9gjpz\") pod \"authorino-59c974bd48-l974l\" (UID: \"fe5995eb-498a-40c7-9176-e6fac60a015d\") " pod="kuadrant-system/authorino-59c974bd48-l974l"
Sep 10 22:14:32.910341 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:32.910307    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l974l"
Sep 10 22:14:33.196331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.196309    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l974l"]
Sep 10 22:14:33.198866 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:14:33.198837    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe5995eb_498a_40c7_9176_e6fac60a015d.slice/crio-6387212d3da851c8245e6e3fd279415d92f9cb074bda1753118665ba82dc605d WatchSource:0}: Error finding container 6387212d3da851c8245e6e3fd279415d92f9cb074bda1753118665ba82dc605d: Status 404 returned error can't find the container with id 6387212d3da851c8245e6e3fd279415d92f9cb074bda1753118665ba82dc605d
Sep 10 22:14:33.339991 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.339959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l974l" event={"ID":"fe5995eb-498a-40c7-9176-e6fac60a015d","Type":"ContainerStarted","Data":"6387212d3da851c8245e6e3fd279415d92f9cb074bda1753118665ba82dc605d"}
Sep 10 22:14:33.341451 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.341416    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs" event={"ID":"502c096c-f9b7-4031-9f45-c8ab5f467fd0","Type":"ContainerStarted","Data":"3be5e635823153129b7e7040dee542b39fa49f7fff18a384584fe7c304b07799"}
Sep 10 22:14:33.341571 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.341485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs" event={"ID":"502c096c-f9b7-4031-9f45-c8ab5f467fd0","Type":"ContainerStarted","Data":"da8343352536ce3e9969a42d50a2a0aecebd8eaf075ca756861af4a538f2e0b6"}
Sep 10 22:14:33.341571 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.341548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:33.343010 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.342985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f" event={"ID":"f3012130-8285-40b0-9906-c2d2e8d8d6a8","Type":"ContainerStarted","Data":"775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b"}
Sep 10 22:14:33.343105 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.343086    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:33.361540 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.361496    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs" podStartSLOduration=1.635631183 podStartE2EDuration="2.361483423s" totalImagesPullingTime="725.85224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:33.360411576 +0000 UTC m=+542.744917000" watchObservedRunningTime="2026-09-10 22:14:33.361483423 +0000 UTC m=+542.745988848"
Sep 10 22:14:33.375802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:33.375763    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f" podStartSLOduration=1.3359315330000001 podStartE2EDuration="2.37574959s" totalImagesPullingTime="1.039818057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:33.375289527 +0000 UTC m=+542.759794952" watchObservedRunningTime="2026-09-10 22:14:33.37574959 +0000 UTC m=+542.760255014"
Sep 10 22:14:35.354565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:35.354529    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l974l" event={"ID":"fe5995eb-498a-40c7-9176-e6fac60a015d","Type":"ContainerStarted","Data":"2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c"}
Sep 10 22:14:35.368191 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:35.368136    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-l974l" podStartSLOduration=2.057293412 podStartE2EDuration="3.36812147s" totalImagesPullingTime="1.310828058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:14:35.367076021 +0000 UTC m=+544.751581450" watchObservedRunningTime="2026-09-10 22:14:35.36812147 +0000 UTC m=+544.752626891"
Sep 10 22:14:44.349805 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.349775    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:44.350317 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.349840    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-d7djs"
Sep 10 22:14:44.406272 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.406244    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-nwj8f"]
Sep 10 22:14:44.406464 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.406432    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f" podUID="f3012130-8285-40b0-9906-c2d2e8d8d6a8" containerName="limitador" containerID="cri-o://775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b" gracePeriod=30
Sep 10 22:14:44.951830 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.951807    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:44.999414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.999381    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t75mf\" (UniqueName: \"kubernetes.io/projected/f3012130-8285-40b0-9906-c2d2e8d8d6a8-kube-api-access-t75mf\") pod \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") "
Sep 10 22:14:44.999593 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.999522    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3012130-8285-40b0-9906-c2d2e8d8d6a8-config-file\") pod \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\" (UID: \"f3012130-8285-40b0-9906-c2d2e8d8d6a8\") "
Sep 10 22:14:44.999862 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:44.999836    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3012130-8285-40b0-9906-c2d2e8d8d6a8-config-file" (OuterVolumeSpecName: "config-file") pod "f3012130-8285-40b0-9906-c2d2e8d8d6a8" (UID: "f3012130-8285-40b0-9906-c2d2e8d8d6a8"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:14:45.001596 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.001565    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3012130-8285-40b0-9906-c2d2e8d8d6a8-kube-api-access-t75mf" (OuterVolumeSpecName: "kube-api-access-t75mf") pod "f3012130-8285-40b0-9906-c2d2e8d8d6a8" (UID: "f3012130-8285-40b0-9906-c2d2e8d8d6a8"). InnerVolumeSpecName "kube-api-access-t75mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:14:45.100512 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.100481    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t75mf\" (UniqueName: \"kubernetes.io/projected/f3012130-8285-40b0-9906-c2d2e8d8d6a8-kube-api-access-t75mf\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:14:45.100512 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.100506    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f3012130-8285-40b0-9906-c2d2e8d8d6a8-config-file\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:14:45.397840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.397750    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3012130-8285-40b0-9906-c2d2e8d8d6a8" containerID="775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b" exitCode=0
Sep 10 22:14:45.397840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.397787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f" event={"ID":"f3012130-8285-40b0-9906-c2d2e8d8d6a8","Type":"ContainerDied","Data":"775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b"}
Sep 10 22:14:45.397840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.397811    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f"
Sep 10 22:14:45.398369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.397851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-nwj8f" event={"ID":"f3012130-8285-40b0-9906-c2d2e8d8d6a8","Type":"ContainerDied","Data":"3e6f22ed4a9cd9859e3b4ad3e35b964422a231a148bbdac632512ea5a844cc12"}
Sep 10 22:14:45.398369 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.397872    2563 scope.go:117] "RemoveContainer" containerID="775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b"
Sep 10 22:14:45.407518 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.407318    2563 scope.go:117] "RemoveContainer" containerID="775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b"
Sep 10 22:14:45.407687 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:14:45.407668    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b\": container with ID starting with 775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b not found: ID does not exist" containerID="775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b"
Sep 10 22:14:45.407741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.407699    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b"} err="failed to get container status \"775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b\": rpc error: code = NotFound desc = could not find container \"775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b\": container with ID starting with 775cefa22405583dbc95dff206b140df5844bb05ea984b6e6921e18408af395b not found: ID does not exist"
Sep 10 22:14:45.412668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.412643    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-nwj8f"]
Sep 10 22:14:45.415858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:45.415833    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-nwj8f"]
Sep 10 22:14:47.110449 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:47.110407    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3012130-8285-40b0-9906-c2d2e8d8d6a8" path="/var/lib/kubelet/pods/f3012130-8285-40b0-9906-c2d2e8d8d6a8/volumes"
Sep 10 22:14:59.177417 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.177383    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f77dd755c-2xzzr"]
Sep 10 22:14:59.177801 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.177733    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3012130-8285-40b0-9906-c2d2e8d8d6a8" containerName="limitador"
Sep 10 22:14:59.177801 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.177745    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3012130-8285-40b0-9906-c2d2e8d8d6a8" containerName="limitador"
Sep 10 22:14:59.177881 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.177809    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3012130-8285-40b0-9906-c2d2e8d8d6a8" containerName="limitador"
Sep 10 22:14:59.180804 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.180785    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.182857 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.182835    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 22:14:59.186535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.186512    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f77dd755c-2xzzr"]
Sep 10 22:14:59.322173 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.322140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xn6t\" (UniqueName: \"kubernetes.io/projected/1b68f76a-8e36-48de-b6b8-4decbc49031c-kube-api-access-7xn6t\") pod \"authorino-f77dd755c-2xzzr\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") " pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.322326 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.322178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b68f76a-8e36-48de-b6b8-4decbc49031c-tls-cert\") pod \"authorino-f77dd755c-2xzzr\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") " pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.422745 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.422715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xn6t\" (UniqueName: \"kubernetes.io/projected/1b68f76a-8e36-48de-b6b8-4decbc49031c-kube-api-access-7xn6t\") pod \"authorino-f77dd755c-2xzzr\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") " pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.422745 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.422745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b68f76a-8e36-48de-b6b8-4decbc49031c-tls-cert\") pod \"authorino-f77dd755c-2xzzr\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") " pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.425227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.425206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b68f76a-8e36-48de-b6b8-4decbc49031c-tls-cert\") pod \"authorino-f77dd755c-2xzzr\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") " pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.430761 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.430694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xn6t\" (UniqueName: \"kubernetes.io/projected/1b68f76a-8e36-48de-b6b8-4decbc49031c-kube-api-access-7xn6t\") pod \"authorino-f77dd755c-2xzzr\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") " pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.490847 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.490803    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:14:59.758172 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:14:59.758141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f77dd755c-2xzzr"]
Sep 10 22:14:59.761069 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:14:59.761040    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b68f76a_8e36_48de_b6b8_4decbc49031c.slice/crio-5831981eb4e37f228c11f2b47bc3bab02b2cbc28ba9ab2331a1e53e454fbb2e0 WatchSource:0}: Error finding container 5831981eb4e37f228c11f2b47bc3bab02b2cbc28ba9ab2331a1e53e454fbb2e0: Status 404 returned error can't find the container with id 5831981eb4e37f228c11f2b47bc3bab02b2cbc28ba9ab2331a1e53e454fbb2e0
Sep 10 22:15:00.462836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:00.462800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f77dd755c-2xzzr" event={"ID":"1b68f76a-8e36-48de-b6b8-4decbc49031c","Type":"ContainerStarted","Data":"5831981eb4e37f228c11f2b47bc3bab02b2cbc28ba9ab2331a1e53e454fbb2e0"}
Sep 10 22:15:01.473804 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.473769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f77dd755c-2xzzr" event={"ID":"1b68f76a-8e36-48de-b6b8-4decbc49031c","Type":"ContainerStarted","Data":"878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7"}
Sep 10 22:15:01.489282 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.489237    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f77dd755c-2xzzr" podStartSLOduration=1.669467392 podStartE2EDuration="2.489218118s" totalImagesPullingTime="819.750726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:15:01.487494114 +0000 UTC m=+570.871999539" watchObservedRunningTime="2026-09-10 22:15:01.489218118 +0000 UTC m=+570.873723542"
Sep 10 22:15:01.516693 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.516664    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l974l"]
Sep 10 22:15:01.516894 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.516871    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-l974l" podUID="fe5995eb-498a-40c7-9176-e6fac60a015d" containerName="authorino" containerID="cri-o://2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c" gracePeriod=30
Sep 10 22:15:01.760124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.760102    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l974l"
Sep 10 22:15:01.842856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.842827    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gjpz\" (UniqueName: \"kubernetes.io/projected/fe5995eb-498a-40c7-9176-e6fac60a015d-kube-api-access-9gjpz\") pod \"fe5995eb-498a-40c7-9176-e6fac60a015d\" (UID: \"fe5995eb-498a-40c7-9176-e6fac60a015d\") "
Sep 10 22:15:01.844941 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.844916    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe5995eb-498a-40c7-9176-e6fac60a015d-kube-api-access-9gjpz" (OuterVolumeSpecName: "kube-api-access-9gjpz") pod "fe5995eb-498a-40c7-9176-e6fac60a015d" (UID: "fe5995eb-498a-40c7-9176-e6fac60a015d"). InnerVolumeSpecName "kube-api-access-9gjpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:15:01.943640 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:01.943611    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gjpz\" (UniqueName: \"kubernetes.io/projected/fe5995eb-498a-40c7-9176-e6fac60a015d-kube-api-access-9gjpz\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:15:02.479219 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.479179    2563 generic.go:356] "Generic (PLEG): container finished" podID="fe5995eb-498a-40c7-9176-e6fac60a015d" containerID="2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c" exitCode=0
Sep 10 22:15:02.479619 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.479223    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-l974l"
Sep 10 22:15:02.479619 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.479259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l974l" event={"ID":"fe5995eb-498a-40c7-9176-e6fac60a015d","Type":"ContainerDied","Data":"2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c"}
Sep 10 22:15:02.479619 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.479302    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-l974l" event={"ID":"fe5995eb-498a-40c7-9176-e6fac60a015d","Type":"ContainerDied","Data":"6387212d3da851c8245e6e3fd279415d92f9cb074bda1753118665ba82dc605d"}
Sep 10 22:15:02.479619 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.479318    2563 scope.go:117] "RemoveContainer" containerID="2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c"
Sep 10 22:15:02.487356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.487338    2563 scope.go:117] "RemoveContainer" containerID="2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c"
Sep 10 22:15:02.487666 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:15:02.487641    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c\": container with ID starting with 2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c not found: ID does not exist" containerID="2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c"
Sep 10 22:15:02.487725 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.487674    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c"} err="failed to get container status \"2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c\": rpc error: code = NotFound desc = could not find container \"2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c\": container with ID starting with 2ae40d8ec59ca898c44db808cd2a8f65a246be9f9e2f5fa36e167ecfa543e15c not found: ID does not exist"
Sep 10 22:15:02.497293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.497272    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l974l"]
Sep 10 22:15:02.499816 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:02.499796    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-l974l"]
Sep 10 22:15:03.110141 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:03.110101    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe5995eb-498a-40c7-9176-e6fac60a015d" path="/var/lib/kubelet/pods/fe5995eb-498a-40c7-9176-e6fac60a015d/volumes"
Sep 10 22:15:46.734376 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.734341    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"]
Sep 10 22:15:46.734779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.734694    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe5995eb-498a-40c7-9176-e6fac60a015d" containerName="authorino"
Sep 10 22:15:46.734779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.734706    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe5995eb-498a-40c7-9176-e6fac60a015d" containerName="authorino"
Sep 10 22:15:46.734779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.734775    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe5995eb-498a-40c7-9176-e6fac60a015d" containerName="authorino"
Sep 10 22:15:46.737222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.737204    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:46.739424 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.739401    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 22:15:46.739566 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.739472    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7qns5\""
Sep 10 22:15:46.745321 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.745301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"]
Sep 10 22:15:46.778235 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.778210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e75642a-f92b-40cf-8223-0902432b07f5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-d88hf\" (UID: \"9e75642a-f92b-40cf-8223-0902432b07f5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:46.778393 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.778301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r94bq\" (UniqueName: \"kubernetes.io/projected/9e75642a-f92b-40cf-8223-0902432b07f5-kube-api-access-r94bq\") pod \"payload-pre-processing-59fbc47bf8-d88hf\" (UID: \"9e75642a-f92b-40cf-8223-0902432b07f5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:46.828107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.828073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:15:46.879629 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.879596    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r94bq\" (UniqueName: \"kubernetes.io/projected/9e75642a-f92b-40cf-8223-0902432b07f5-kube-api-access-r94bq\") pod \"payload-pre-processing-59fbc47bf8-d88hf\" (UID: \"9e75642a-f92b-40cf-8223-0902432b07f5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:46.879822 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.879641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e75642a-f92b-40cf-8223-0902432b07f5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-d88hf\" (UID: \"9e75642a-f92b-40cf-8223-0902432b07f5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:46.880483 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.880432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e75642a-f92b-40cf-8223-0902432b07f5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-d88hf\" (UID: \"9e75642a-f92b-40cf-8223-0902432b07f5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:46.892130 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:46.892105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r94bq\" (UniqueName: \"kubernetes.io/projected/9e75642a-f92b-40cf-8223-0902432b07f5-kube-api-access-r94bq\") pod \"payload-pre-processing-59fbc47bf8-d88hf\" (UID: \"9e75642a-f92b-40cf-8223-0902432b07f5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:47.047735 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:47.047654    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:47.318312 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:47.318288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"]
Sep 10 22:15:47.320689 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:15:47.320660    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e75642a_f92b_40cf_8223_0902432b07f5.slice/crio-c8c183a7dc2d1713465bf3e76068bed29e41783d8c28de00f9b64f5a6205b5a6 WatchSource:0}: Error finding container c8c183a7dc2d1713465bf3e76068bed29e41783d8c28de00f9b64f5a6205b5a6: Status 404 returned error can't find the container with id c8c183a7dc2d1713465bf3e76068bed29e41783d8c28de00f9b64f5a6205b5a6
Sep 10 22:15:47.677400 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:47.677317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf" event={"ID":"9e75642a-f92b-40cf-8223-0902432b07f5","Type":"ContainerStarted","Data":"c8c183a7dc2d1713465bf3e76068bed29e41783d8c28de00f9b64f5a6205b5a6"}
Sep 10 22:15:50.693801 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:50.693765    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf" event={"ID":"9e75642a-f92b-40cf-8223-0902432b07f5","Type":"ContainerStarted","Data":"7f660c2d97f0835f72c9a51ad34febc14a87e362d4a6870612ac59b1792b9dc4"}
Sep 10 22:15:50.694288 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:50.693878    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:15:56.721914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:56.721880    2563 generic.go:356] "Generic (PLEG): container finished" podID="1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec" containerID="30e7232d27128a30601eff503ee6c249c5a76cfe47a874f5c0d44453eaa8c930" exitCode=0
Sep 10 22:15:56.722331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:56.721936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" event={"ID":"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec","Type":"ContainerDied","Data":"30e7232d27128a30601eff503ee6c249c5a76cfe47a874f5c0d44453eaa8c930"}
Sep 10 22:15:56.722331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:56.722308    2563 scope.go:117] "RemoveContainer" containerID="30e7232d27128a30601eff503ee6c249c5a76cfe47a874f5c0d44453eaa8c930"
Sep 10 22:15:56.723225 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:56.723200    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:15:56.723284 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:56.723265    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:15:56.723321 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:56.723298    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:15:56.741234 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:56.741182    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf" podStartSLOduration=8.455073468 podStartE2EDuration="10.741162296s" totalImagesPullingTime="2.286088828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:15:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:15:50.721870928 +0000 UTC m=+620.106376355" watchObservedRunningTime="2026-09-10 22:15:56.741162296 +0000 UTC m=+626.125667722"
Sep 10 22:15:57.732611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:57.732562    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" event={"ID":"1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec","Type":"ContainerStarted","Data":"b8649e10e14f657b5bf2e79f1e3f580e785537b1248d30c6af9aa8d3a30c1b7c"}
Sep 10 22:15:58.095409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:58.095321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:15:58.096486 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:58.096456    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-2784q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 22:15:58.096602 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:58.096509    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" podUID="1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 22:15:59.095653 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:59.095619    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-2784q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 22:15:59.096030 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:15:59.095675    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" podUID="1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 22:16:00.095478 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.095418    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-2784q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 22:16:00.095654 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.095496    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" podUID="1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 22:16:00.323359 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.323312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:16:00.904948 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.904920    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"]
Sep 10 22:16:00.907250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.907234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:00.910291 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.910265    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 22:16:00.910427 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.910320    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 22:16:00.910427 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.910279    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-v5j2d\""
Sep 10 22:16:00.910427 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.910270    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 22:16:00.919465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:00.919251    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"]
Sep 10 22:16:01.010364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.010335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e57b7c-fa1f-4330-9ae3-ce1676d51530-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.010569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.010375    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.010569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.010401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.010569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.010532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.010704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.010608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfkc8\" (UniqueName: \"kubernetes.io/projected/81e57b7c-fa1f-4330-9ae3-ce1676d51530-kube-api-access-gfkc8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.010704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.010648    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.095531 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.095495    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-2784q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 22:16:01.095739 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.095556    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" podUID="1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 22:16:01.111010 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.110985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfkc8\" (UniqueName: \"kubernetes.io/projected/81e57b7c-fa1f-4330-9ae3-ce1676d51530-kube-api-access-gfkc8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111145    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e57b7c-fa1f-4330-9ae3-ce1676d51530-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111295 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111295 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111378 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.111624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.111613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.116704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.114402    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81e57b7c-fa1f-4330-9ae3-ce1676d51530-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.116833 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.116708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81e57b7c-fa1f-4330-9ae3-ce1676d51530-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.121053 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.121028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfkc8\" (UniqueName: \"kubernetes.io/projected/81e57b7c-fa1f-4330-9ae3-ce1676d51530-kube-api-access-gfkc8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6\" (UID: \"81e57b7c-fa1f-4330-9ae3-ce1676d51530\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.239000 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.238751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:01.492827 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.492726    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:16:01.515321 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.515286    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"]
Sep 10 22:16:01.521803 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:16:01.521754    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e57b7c_fa1f_4330_9ae3_ce1676d51530.slice/crio-04215bce2573210e83b0022aa4982ec520ebd8e628167f26b7c02c28c4bb094d WatchSource:0}: Error finding container 04215bce2573210e83b0022aa4982ec520ebd8e628167f26b7c02c28c4bb094d: Status 404 returned error can't find the container with id 04215bce2573210e83b0022aa4982ec520ebd8e628167f26b7c02c28c4bb094d
Sep 10 22:16:01.700929 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.700892    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-d88hf"
Sep 10 22:16:01.754981 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:01.754878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6" event={"ID":"81e57b7c-fa1f-4330-9ae3-ce1676d51530","Type":"ContainerStarted","Data":"04215bce2573210e83b0022aa4982ec520ebd8e628167f26b7c02c28c4bb094d"}
Sep 10 22:16:02.096830 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:02.096743    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-2784q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 22:16:02.096830 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:02.096810    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q" podUID="1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 22:16:03.098856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:03.098790    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:16:03.099671 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:03.099654    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2784q"
Sep 10 22:16:03.766214 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:03.766174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6" event={"ID":"81e57b7c-fa1f-4330-9ae3-ce1676d51530","Type":"ContainerStarted","Data":"62d168f0d6505fe910d14f67e8de2db4660f7c61d78bcaf24a5cd3db194cc4a4"}
Sep 10 22:16:03.783906 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:03.783855    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6" podStartSLOduration=2.499423303 podStartE2EDuration="3.783840029s" totalImagesPullingTime="1.284416726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:16:03.781749138 +0000 UTC m=+633.166254575" watchObservedRunningTime="2026-09-10 22:16:03.783840029 +0000 UTC m=+633.168345455"
Sep 10 22:16:08.093679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:08.093644    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:16:11.239747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:11.239717    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:11.240167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:11.239791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:11.252109 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:11.252084    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:11.812866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:11.812841    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6"
Sep 10 22:16:30.100632 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:30.100597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:16:37.509165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.509133    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"]
Sep 10 22:16:37.513401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.513383    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.515428 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.515400    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 22:16:37.523045 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.523024    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"]
Sep 10 22:16:37.633310 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.633274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvf9h\" (UniqueName: \"kubernetes.io/projected/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-kube-api-access-rvf9h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.633503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.633316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.633503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.633341    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.633503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.633467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.633503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.633499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.633705 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.633554    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.734795 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.734756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.734990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.734815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvf9h\" (UniqueName: \"kubernetes.io/projected/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-kube-api-access-rvf9h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.734990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.734846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.734990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.734870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.734990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.734937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.734990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.734967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.735317 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.735289    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.735380 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.735312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.735380 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.735329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.737125 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.737106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.737527 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.737508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.742216 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.742198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvf9h\" (UniqueName: \"kubernetes.io/projected/ffad8d35-0731-4da0-8bc7-451d5f8a2ff5-kube-api-access-rvf9h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-66h7v\" (UID: \"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:37.824431 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:37.824341    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:38.094906 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:38.094825    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"]
Sep 10 22:16:38.097696 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:16:38.097664    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffad8d35_0731_4da0_8bc7_451d5f8a2ff5.slice/crio-688c6a36ae417b4ecc896c7f20422f25d02a17b7bd50ba5869dda96e768c3d3f WatchSource:0}: Error finding container 688c6a36ae417b4ecc896c7f20422f25d02a17b7bd50ba5869dda96e768c3d3f: Status 404 returned error can't find the container with id 688c6a36ae417b4ecc896c7f20422f25d02a17b7bd50ba5869dda96e768c3d3f
Sep 10 22:16:38.098987 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:38.098968    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 22:16:38.712594 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:38.712558    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:16:38.922522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:38.922485    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v" event={"ID":"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5","Type":"ContainerStarted","Data":"b8212813e5bb2f820f4334dac96fbb86daab8b615673a870c00559099ff5a1ae"}
Sep 10 22:16:38.922727 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:38.922551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v" event={"ID":"ffad8d35-0731-4da0-8bc7-451d5f8a2ff5","Type":"ContainerStarted","Data":"688c6a36ae417b4ecc896c7f20422f25d02a17b7bd50ba5869dda96e768c3d3f"}
Sep 10 22:16:38.941232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:38.941179    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v" podStartSLOduration=1.761356623 podStartE2EDuration="1.941161485s" totalImagesPullingTime="179.804862ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:16:38.940055317 +0000 UTC m=+668.324560743" watchObservedRunningTime="2026-09-10 22:16:38.941161485 +0000 UTC m=+668.325666909"
Sep 10 22:16:47.825112 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:47.825077    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:47.825551 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:47.825167    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:47.837746 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:47.837720    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:47.975324 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:47.975295    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-66h7v"
Sep 10 22:16:52.097283 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:16:52.097238    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:17:07.808003 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:07.807967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:17:13.401946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.401911    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"]
Sep 10 22:17:13.405276 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.405254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.407407 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.407383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 22:17:13.415980 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.415952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"]
Sep 10 22:17:13.429305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.429274    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cc432e1-0fe2-44a4-93ce-a507a67751a7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.429465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.429311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.429465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.429364    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.429465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.429382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr7lt\" (UniqueName: \"kubernetes.io/projected/4cc432e1-0fe2-44a4-93ce-a507a67751a7-kube-api-access-xr7lt\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.429465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.429404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.429607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.429490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.530536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.530497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.530702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.530548    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cc432e1-0fe2-44a4-93ce-a507a67751a7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.530702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.530606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.530702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.530680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.530852 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.530715    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr7lt\" (UniqueName: \"kubernetes.io/projected/4cc432e1-0fe2-44a4-93ce-a507a67751a7-kube-api-access-xr7lt\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.530852 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.530753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.531254 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.531225    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.531630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.531255    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.531630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.531313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.533122 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.533094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4cc432e1-0fe2-44a4-93ce-a507a67751a7-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.533206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.533163    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4cc432e1-0fe2-44a4-93ce-a507a67751a7-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.538301 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.538283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr7lt\" (UniqueName: \"kubernetes.io/projected/4cc432e1-0fe2-44a4-93ce-a507a67751a7-kube-api-access-xr7lt\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7\" (UID: \"4cc432e1-0fe2-44a4-93ce-a507a67751a7\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.716174 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.716081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:13.985978 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:13.985953    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"]
Sep 10 22:17:13.987866 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:17:13.987834    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4cc432e1_0fe2_44a4_93ce_a507a67751a7.slice/crio-450fd7d6af84dc1f2a98d738c7b0734e3ea6734b5b77298ae65b1b751a6e13a7 WatchSource:0}: Error finding container 450fd7d6af84dc1f2a98d738c7b0734e3ea6734b5b77298ae65b1b751a6e13a7: Status 404 returned error can't find the container with id 450fd7d6af84dc1f2a98d738c7b0734e3ea6734b5b77298ae65b1b751a6e13a7
Sep 10 22:17:14.083627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:14.083596    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7" event={"ID":"4cc432e1-0fe2-44a4-93ce-a507a67751a7","Type":"ContainerStarted","Data":"450fd7d6af84dc1f2a98d738c7b0734e3ea6734b5b77298ae65b1b751a6e13a7"}
Sep 10 22:17:15.089280 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:15.089248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7" event={"ID":"4cc432e1-0fe2-44a4-93ce-a507a67751a7","Type":"ContainerStarted","Data":"9b67e3a00a74c25b88adc210a094fae2de663e50f174ef07b9ea0e1a444cb2f4"}
Sep 10 22:17:15.104007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:15.103952    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7" podStartSLOduration=1.92940117 podStartE2EDuration="2.103931886s" totalImagesPullingTime="174.530716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:17:15.103262331 +0000 UTC m=+704.487767758" watchObservedRunningTime="2026-09-10 22:17:15.103931886 +0000 UTC m=+704.488437308"
Sep 10 22:17:15.605836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:15.605803    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:17:23.716346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:23.716266    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:23.716346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:23.716318    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:23.728716 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:23.728683    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:17:24.139653 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:17:24.139625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7"
Sep 10 22:18:45.920615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:45.920579    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f77dd755c-2xzzr"]
Sep 10 22:18:45.921185 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:45.920815    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-f77dd755c-2xzzr" podUID="1b68f76a-8e36-48de-b6b8-4decbc49031c" containerName="authorino" containerID="cri-o://878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7" gracePeriod=30
Sep 10 22:18:46.168528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.168505    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:18:46.332377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.332346    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b68f76a-8e36-48de-b6b8-4decbc49031c-tls-cert\") pod \"1b68f76a-8e36-48de-b6b8-4decbc49031c\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") "
Sep 10 22:18:46.332599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.332411    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xn6t\" (UniqueName: \"kubernetes.io/projected/1b68f76a-8e36-48de-b6b8-4decbc49031c-kube-api-access-7xn6t\") pod \"1b68f76a-8e36-48de-b6b8-4decbc49031c\" (UID: \"1b68f76a-8e36-48de-b6b8-4decbc49031c\") "
Sep 10 22:18:46.334611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.334580    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b68f76a-8e36-48de-b6b8-4decbc49031c-kube-api-access-7xn6t" (OuterVolumeSpecName: "kube-api-access-7xn6t") pod "1b68f76a-8e36-48de-b6b8-4decbc49031c" (UID: "1b68f76a-8e36-48de-b6b8-4decbc49031c"). InnerVolumeSpecName "kube-api-access-7xn6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:18:46.342867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.342839    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b68f76a-8e36-48de-b6b8-4decbc49031c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1b68f76a-8e36-48de-b6b8-4decbc49031c" (UID: "1b68f76a-8e36-48de-b6b8-4decbc49031c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:18:46.433942 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.433904    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b68f76a-8e36-48de-b6b8-4decbc49031c-tls-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:18:46.433942 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.433940    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xn6t\" (UniqueName: \"kubernetes.io/projected/1b68f76a-8e36-48de-b6b8-4decbc49031c-kube-api-access-7xn6t\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:18:46.500645 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.500611    2563 generic.go:356] "Generic (PLEG): container finished" podID="1b68f76a-8e36-48de-b6b8-4decbc49031c" containerID="878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7" exitCode=0
Sep 10 22:18:46.500786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.500658    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f77dd755c-2xzzr"
Sep 10 22:18:46.500786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.500705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f77dd755c-2xzzr" event={"ID":"1b68f76a-8e36-48de-b6b8-4decbc49031c","Type":"ContainerDied","Data":"878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7"}
Sep 10 22:18:46.500786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.500745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f77dd755c-2xzzr" event={"ID":"1b68f76a-8e36-48de-b6b8-4decbc49031c","Type":"ContainerDied","Data":"5831981eb4e37f228c11f2b47bc3bab02b2cbc28ba9ab2331a1e53e454fbb2e0"}
Sep 10 22:18:46.500786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.500763    2563 scope.go:117] "RemoveContainer" containerID="878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7"
Sep 10 22:18:46.509494 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.509476    2563 scope.go:117] "RemoveContainer" containerID="878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7"
Sep 10 22:18:46.509808 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:18:46.509789    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7\": container with ID starting with 878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7 not found: ID does not exist" containerID="878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7"
Sep 10 22:18:46.509872 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.509817    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7"} err="failed to get container status \"878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7\": rpc error: code = NotFound desc = could not find container \"878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7\": container with ID starting with 878224263e7de2bfc6b87efa5fd9fb9f4eb8c7c83a9058111e50260cb40f26d7 not found: ID does not exist"
Sep 10 22:18:46.520184 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.520158    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f77dd755c-2xzzr"]
Sep 10 22:18:46.523829 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:46.523805    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f77dd755c-2xzzr"]
Sep 10 22:18:47.111039 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:18:47.111011    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b68f76a-8e36-48de-b6b8-4decbc49031c" path="/var/lib/kubelet/pods/1b68f76a-8e36-48de-b6b8-4decbc49031c/volumes"
Sep 10 22:19:03.150230 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.150148    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"]
Sep 10 22:19:03.150592 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.150549    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b68f76a-8e36-48de-b6b8-4decbc49031c" containerName="authorino"
Sep 10 22:19:03.150592 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.150563    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b68f76a-8e36-48de-b6b8-4decbc49031c" containerName="authorino"
Sep 10 22:19:03.150668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.150628    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b68f76a-8e36-48de-b6b8-4decbc49031c" containerName="authorino"
Sep 10 22:19:03.153524 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.153504    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.156500 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.156470    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-669e2b-openshift-default-dockercfg-npxpp\""
Sep 10 22:19:03.164793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.164772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-podinfo\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.164886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.164821    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-socket\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.164886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.164857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-envoy\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.164886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.164876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56487366-5ec6-41d9-afd0-9534f65f1b3b-istiod-ca-cert\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.165025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.164902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-data\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.165025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.164937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf58s\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-kube-api-access-hf58s\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.165025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.164953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-certs\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.165177 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.165065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-token\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.165177 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.165121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-credential-socket\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.169672 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.169618    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"]
Sep 10 22:19:03.265879 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.265836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf58s\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-kube-api-access-hf58s\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.265879 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.265883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-certs\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266092 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.265942    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-token\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266092 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.265965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-credential-socket\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266092 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.265982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-podinfo\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.266128    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-socket\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.266182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-envoy\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.266240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56487366-5ec6-41d9-afd0-9534f65f1b3b-istiod-ca-cert\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.266280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-data\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.266321    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-certs\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.266599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-credential-socket\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.266715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.266693    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-data\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.267108 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.267084    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-socket\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.267293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.267269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56487366-5ec6-41d9-afd0-9534f65f1b3b-istiod-ca-cert\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.268683 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.268664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-envoy\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.268764 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.268687    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-podinfo\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.273969 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.273944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-token\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.273969 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.273965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf58s\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-kube-api-access-hf58s\") pod \"e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") " pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.464467 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.464351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:03.756410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.756248    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"]
Sep 10 22:19:03.759164 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:03.759130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56487366_5ec6_41d9_afd0_9534f65f1b3b.slice/crio-97ef8a3efddd27b53ab0f71bf0565ad8240406662c176e71143d302e177893c9 WatchSource:0}: Error finding container 97ef8a3efddd27b53ab0f71bf0565ad8240406662c176e71143d302e177893c9: Status 404 returned error can't find the container with id 97ef8a3efddd27b53ab0f71bf0565ad8240406662c176e71143d302e177893c9
Sep 10 22:19:03.761350 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.761316    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:03.761479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.761383    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:03.761479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:03.761410    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:04.585248 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.585201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" event={"ID":"56487366-5ec6-41d9-afd0-9534f65f1b3b","Type":"ContainerStarted","Data":"9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12"}
Sep 10 22:19:04.585746 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.585255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" event={"ID":"56487366-5ec6-41d9-afd0-9534f65f1b3b","Type":"ContainerStarted","Data":"97ef8a3efddd27b53ab0f71bf0565ad8240406662c176e71143d302e177893c9"}
Sep 10 22:19:04.608119 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.608070    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podStartSLOduration=1.608053361 podStartE2EDuration="1.608053361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:04.606490483 +0000 UTC m=+813.990995909" watchObservedRunningTime="2026-09-10 22:19:04.608053361 +0000 UTC m=+813.992558785"
Sep 10 22:19:04.868154 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.868080    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7617-f40ddb"]
Sep 10 22:19:04.871430 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.871413    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:04.873812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.873793    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qckd6\""
Sep 10 22:19:04.878041 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.878019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbwd8\" (UniqueName: \"kubernetes.io/projected/e3a4b037-4706-42ef-9a3d-e3ae1e9bc547-kube-api-access-gbwd8\") pod \"test-curl-7617-f40ddb\" (UID: \"e3a4b037-4706-42ef-9a3d-e3ae1e9bc547\") " pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:04.893858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.893832    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7617-f40ddb"]
Sep 10 22:19:04.978490 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.978459    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbwd8\" (UniqueName: \"kubernetes.io/projected/e3a4b037-4706-42ef-9a3d-e3ae1e9bc547-kube-api-access-gbwd8\") pod \"test-curl-7617-f40ddb\" (UID: \"e3a4b037-4706-42ef-9a3d-e3ae1e9bc547\") " pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:04.986391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:04.986361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbwd8\" (UniqueName: \"kubernetes.io/projected/e3a4b037-4706-42ef-9a3d-e3ae1e9bc547-kube-api-access-gbwd8\") pod \"test-curl-7617-f40ddb\" (UID: \"e3a4b037-4706-42ef-9a3d-e3ae1e9bc547\") " pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:05.180371 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:05.180277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:05.451530 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:05.451487    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7617-f40ddb"]
Sep 10 22:19:05.453987 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:05.453960    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3a4b037_4706_42ef_9a3d_e3ae1e9bc547.slice/crio-fdeba4f1cc687cd0d2638783b69184cfe665bdadb26b83436a314f7d6f310a0c WatchSource:0}: Error finding container fdeba4f1cc687cd0d2638783b69184cfe665bdadb26b83436a314f7d6f310a0c: Status 404 returned error can't find the container with id fdeba4f1cc687cd0d2638783b69184cfe665bdadb26b83436a314f7d6f310a0c
Sep 10 22:19:05.465295 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:05.465276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:05.470075 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:05.470055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:05.591485 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:05.591420    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7617-f40ddb" event={"ID":"e3a4b037-4706-42ef-9a3d-e3ae1e9bc547","Type":"ContainerStarted","Data":"fdeba4f1cc687cd0d2638783b69184cfe665bdadb26b83436a314f7d6f310a0c"}
Sep 10 22:19:05.591861 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:05.591648    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:05.592747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:05.592728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:06.398697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:06.398660    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:19:08.511766 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.511730    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"]
Sep 10 22:19:08.515171 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.515155    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:08.517375 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.517348    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-669e2b\""
Sep 10 22:19:08.517504 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.517348    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-669e2b-dockercfg-xdn5m\""
Sep 10 22:19:08.525366 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.525346    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"]
Sep 10 22:19:08.606380 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.606351    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42lqk\" (UniqueName: \"kubernetes.io/projected/85021748-c50c-4fa8-ae10-760bb28db67e-kube-api-access-42lqk\") pod \"payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:08.606569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.606531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85021748-c50c-4fa8-ae10-760bb28db67e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:08.607896 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.607869    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7617-f40ddb" event={"ID":"e3a4b037-4706-42ef-9a3d-e3ae1e9bc547","Type":"ContainerStarted","Data":"00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba"}
Sep 10 22:19:08.622942 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.622877    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7617-f40ddb" podStartSLOduration=2.372585114 podStartE2EDuration="4.622861495s" totalImagesPullingTime="2.250276381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:08.622213278 +0000 UTC m=+818.006718704" watchObservedRunningTime="2026-09-10 22:19:08.622861495 +0000 UTC m=+818.007366920"
Sep 10 22:19:08.707674 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.707642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85021748-c50c-4fa8-ae10-760bb28db67e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:08.707859 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.707683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42lqk\" (UniqueName: \"kubernetes.io/projected/85021748-c50c-4fa8-ae10-760bb28db67e-kube-api-access-42lqk\") pod \"payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:08.708515 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.708494    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85021748-c50c-4fa8-ae10-760bb28db67e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:08.715893 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.715869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42lqk\" (UniqueName: \"kubernetes.io/projected/85021748-c50c-4fa8-ae10-760bb28db67e-kube-api-access-42lqk\") pod \"payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:08.826550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:08.826505    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:09.095867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.095844    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"]
Sep 10 22:19:09.097782 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:09.097756    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85021748_c50c_4fa8_ae10_760bb28db67e.slice/crio-2a3cad77e3bd978a49c9fa0c4559793cc74b74c202cbb7b3f2988441a9c9ed92 WatchSource:0}: Error finding container 2a3cad77e3bd978a49c9fa0c4559793cc74b74c202cbb7b3f2988441a9c9ed92: Status 404 returned error can't find the container with id 2a3cad77e3bd978a49c9fa0c4559793cc74b74c202cbb7b3f2988441a9c9ed92
Sep 10 22:19:09.199211 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.199176    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7617-f40ddb"]
Sep 10 22:19:09.203448 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.203400    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7617-f40ddb"]
Sep 10 22:19:09.322484 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.322449    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7617-572958"]
Sep 10 22:19:09.322806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.322794    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" containerName="test-curl-7617-f40ddb"
Sep 10 22:19:09.322858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.322807    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" containerName="test-curl-7617-f40ddb"
Sep 10 22:19:09.322858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.322857    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" containerName="test-curl-7617-f40ddb"
Sep 10 22:19:09.325950 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.325930    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:09.331222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.331201    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7617-572958"]
Sep 10 22:19:09.414153 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.414069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj92w\" (UniqueName: \"kubernetes.io/projected/34242533-f09d-4ada-9b53-6ee637edd6f5-kube-api-access-mj92w\") pod \"test-curl-7617-572958\" (UID: \"34242533-f09d-4ada-9b53-6ee637edd6f5\") " pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:09.514821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.514782    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj92w\" (UniqueName: \"kubernetes.io/projected/34242533-f09d-4ada-9b53-6ee637edd6f5-kube-api-access-mj92w\") pod \"test-curl-7617-572958\" (UID: \"34242533-f09d-4ada-9b53-6ee637edd6f5\") " pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:09.522690 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.522665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj92w\" (UniqueName: \"kubernetes.io/projected/34242533-f09d-4ada-9b53-6ee637edd6f5-kube-api-access-mj92w\") pod \"test-curl-7617-572958\" (UID: \"34242533-f09d-4ada-9b53-6ee637edd6f5\") " pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:09.614523 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.614489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb" event={"ID":"85021748-c50c-4fa8-ae10-760bb28db67e","Type":"ContainerStarted","Data":"8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a"}
Sep 10 22:19:09.614675 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.614550    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb" event={"ID":"85021748-c50c-4fa8-ae10-760bb28db67e","Type":"ContainerStarted","Data":"2a3cad77e3bd978a49c9fa0c4559793cc74b74c202cbb7b3f2988441a9c9ed92"}
Sep 10 22:19:09.614733 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:09.614681    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7617-f40ddb" container="test-curl-7617-f40ddb"
Sep 10 22:19:09.630921 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.630874    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb" podStartSLOduration=1.630858642 podStartE2EDuration="1.630858642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:09.629668277 +0000 UTC m=+819.014173702" watchObservedRunningTime="2026-09-10 22:19:09.630858642 +0000 UTC m=+819.015364067"
Sep 10 22:19:09.635874 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.635851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:09.905210 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:09.905102    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7617-572958"]
Sep 10 22:19:09.907622 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:09.907595    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34242533_f09d_4ada_9b53_6ee637edd6f5.slice/crio-3cc487f5a0345ba6fa1fa1b31d03ed102ee72749f0822dd5d839cc9edc5bcb61 WatchSource:0}: Error finding container 3cc487f5a0345ba6fa1fa1b31d03ed102ee72749f0822dd5d839cc9edc5bcb61: Status 404 returned error can't find the container with id 3cc487f5a0345ba6fa1fa1b31d03ed102ee72749f0822dd5d839cc9edc5bcb61
Sep 10 22:19:10.621176 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:10.621135    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7617-572958" event={"ID":"34242533-f09d-4ada-9b53-6ee637edd6f5","Type":"ContainerStarted","Data":"cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4"}
Sep 10 22:19:10.621658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:10.621187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7617-572958" event={"ID":"34242533-f09d-4ada-9b53-6ee637edd6f5","Type":"ContainerStarted","Data":"3cc487f5a0345ba6fa1fa1b31d03ed102ee72749f0822dd5d839cc9edc5bcb61"}
Sep 10 22:19:10.621658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:10.621219    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7617-f40ddb" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" containerName="test-curl-7617-f40ddb" containerID="cri-o://00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba" gracePeriod=2
Sep 10 22:19:10.621658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:10.621530    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:10.623390 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:10.623365    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7617-f40ddb\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:10.638277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:10.638230    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7617-572958" podStartSLOduration=1.6382085659999999 podStartE2EDuration="1.638208566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:10.637750867 +0000 UTC m=+820.022256292" watchObservedRunningTime="2026-09-10 22:19:10.638208566 +0000 UTC m=+820.022713993"
Sep 10 22:19:10.639827 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:10.639795    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7617-f40ddb\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:11.109218 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:11.109148    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7617-f40ddb\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:11.248617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:11.248589    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7617-572958"]
Sep 10 22:19:11.253771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:11.253680    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7617-572958"]
Sep 10 22:19:11.627031 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:11.626997    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7617-572958" container="test-curl-7617-572958"
Sep 10 22:19:12.054474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.049238    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"]
Sep 10 22:19:12.054474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.051271    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34242533-f09d-4ada-9b53-6ee637edd6f5" containerName="test-curl-7617-572958"
Sep 10 22:19:12.054474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.051294    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34242533-f09d-4ada-9b53-6ee637edd6f5" containerName="test-curl-7617-572958"
Sep 10 22:19:12.054474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.051412    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34242533-f09d-4ada-9b53-6ee637edd6f5" containerName="test-curl-7617-572958"
Sep 10 22:19:12.056300 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.054895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.058037 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.058007    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-44dc54-openshift-default-dockercfg-nksjf\""
Sep 10 22:19:12.071683 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.071655    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"]
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.138909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-data\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx6p4\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-kube-api-access-rx6p4\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istiod-ca-cert\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-podinfo\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-token\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-certs\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-envoy\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139231    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-socket\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.140182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.139270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-credential-socket\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-podinfo\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-token\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-certs\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-envoy\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-socket\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-credential-socket\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-data\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx6p4\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-kube-api-access-rx6p4\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.240461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.240454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istiod-ca-cert\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.241276 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.241249    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istiod-ca-cert\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.242185 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.242162    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-socket\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.243102 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.243052    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-credential-socket\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.243474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.243453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-certs\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.243902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.243883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-data\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.246976 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.246949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-podinfo\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.250152 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.250099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-envoy\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.251071 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.251045    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx6p4\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-kube-api-access-rx6p4\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.251902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.251879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-token\") pod \"e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") " pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.370689 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.370598    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:12.644491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.643724    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7617-572958" podUID="34242533-f09d-4ada-9b53-6ee637edd6f5" containerName="test-curl-7617-572958" containerID="cri-o://cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4" gracePeriod=2
Sep 10 22:19:12.646423 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:12.646370    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7617-572958\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="34242533-f09d-4ada-9b53-6ee637edd6f5" pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:12.696120 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.696084    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"]
Sep 10 22:19:12.701451 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:12.701411    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod129e09c3_9ae9_4c39_9493_ea5a2654b5aa.slice/crio-c9eda03f4ae372d72c2dc48e1a8dba857e98dda62504db76c7df4612c5b803dd WatchSource:0}: Error finding container c9eda03f4ae372d72c2dc48e1a8dba857e98dda62504db76c7df4612c5b803dd: Status 404 returned error can't find the container with id c9eda03f4ae372d72c2dc48e1a8dba857e98dda62504db76c7df4612c5b803dd
Sep 10 22:19:12.704250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.704214    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:12.704322 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.704292    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:12.704362 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.704350    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:12.898920 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.898699    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:12.900755 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:12.900724    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7617-572958\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="34242533-f09d-4ada-9b53-6ee637edd6f5" pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:12.961466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.957855    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbwd8\" (UniqueName: \"kubernetes.io/projected/e3a4b037-4706-42ef-9a3d-e3ae1e9bc547-kube-api-access-gbwd8\") pod \"e3a4b037-4706-42ef-9a3d-e3ae1e9bc547\" (UID: \"e3a4b037-4706-42ef-9a3d-e3ae1e9bc547\") "
Sep 10 22:19:12.965776 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:12.965700    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3a4b037-4706-42ef-9a3d-e3ae1e9bc547-kube-api-access-gbwd8" (OuterVolumeSpecName: "kube-api-access-gbwd8") pod "e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" (UID: "e3a4b037-4706-42ef-9a3d-e3ae1e9bc547"). InnerVolumeSpecName "kube-api-access-gbwd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:19:13.059307 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.059275    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbwd8\" (UniqueName: \"kubernetes.io/projected/e3a4b037-4706-42ef-9a3d-e3ae1e9bc547-kube-api-access-gbwd8\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:19:13.110613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.110582    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" path="/var/lib/kubelet/pods/e3a4b037-4706-42ef-9a3d-e3ae1e9bc547/volumes"
Sep 10 22:19:13.650618 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.650519    2563 generic.go:356] "Generic (PLEG): container finished" podID="e3a4b037-4706-42ef-9a3d-e3ae1e9bc547" containerID="00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba" exitCode=137
Sep 10 22:19:13.650618 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.650575    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-f40ddb"
Sep 10 22:19:13.650618 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.650607    2563 scope.go:117] "RemoveContainer" containerID="00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba"
Sep 10 22:19:13.652605 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.652574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" event={"ID":"129e09c3-9ae9-4c39-9493-ea5a2654b5aa","Type":"ContainerStarted","Data":"ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31"}
Sep 10 22:19:13.652716 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.652636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" event={"ID":"129e09c3-9ae9-4c39-9493-ea5a2654b5aa","Type":"ContainerStarted","Data":"c9eda03f4ae372d72c2dc48e1a8dba857e98dda62504db76c7df4612c5b803dd"}
Sep 10 22:19:13.659730 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.659711    2563 scope.go:117] "RemoveContainer" containerID="00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba"
Sep 10 22:19:13.659998 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:13.659981    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba\": container with ID starting with 00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba not found: ID does not exist" containerID="00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba"
Sep 10 22:19:13.660079 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.660005    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba"} err="failed to get container status \"00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba\": rpc error: code = NotFound desc = could not find container \"00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba\": container with ID starting with 00e87333f95d69b6920b10b857345a46c1b3fd4e59f96091b10cce7da2701cba not found: ID does not exist"
Sep 10 22:19:13.674477 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:13.674410    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podStartSLOduration=1.674394471 podStartE2EDuration="1.674394471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:13.673116412 +0000 UTC m=+823.057621837" watchObservedRunningTime="2026-09-10 22:19:13.674394471 +0000 UTC m=+823.058899896"
Sep 10 22:19:14.370944 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:14.370908    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:14.375804 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:14.375775    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:14.659458 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:14.659387    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:14.660496 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:14.660466    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:14.866544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:14.866514    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:14.977367 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:14.977285    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj92w\" (UniqueName: \"kubernetes.io/projected/34242533-f09d-4ada-9b53-6ee637edd6f5-kube-api-access-mj92w\") pod \"34242533-f09d-4ada-9b53-6ee637edd6f5\" (UID: \"34242533-f09d-4ada-9b53-6ee637edd6f5\") "
Sep 10 22:19:14.979522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:14.979492    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34242533-f09d-4ada-9b53-6ee637edd6f5-kube-api-access-mj92w" (OuterVolumeSpecName: "kube-api-access-mj92w") pod "34242533-f09d-4ada-9b53-6ee637edd6f5" (UID: "34242533-f09d-4ada-9b53-6ee637edd6f5"). InnerVolumeSpecName "kube-api-access-mj92w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:19:15.078743 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.078715    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj92w\" (UniqueName: \"kubernetes.io/projected/34242533-f09d-4ada-9b53-6ee637edd6f5-kube-api-access-mj92w\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:19:15.110448 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.110406    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34242533-f09d-4ada-9b53-6ee637edd6f5" path="/var/lib/kubelet/pods/34242533-f09d-4ada-9b53-6ee637edd6f5/volumes"
Sep 10 22:19:15.239806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.239773    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"]
Sep 10 22:19:15.244123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.244098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"]
Sep 10 22:19:15.244244 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.244189    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.246533 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.246512    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-938f94d8\""
Sep 10 22:19:15.246684 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.246512    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-938f94d8-dockercfg-rqmwm\""
Sep 10 22:19:15.280983 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.280953    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q48z6\" (UniqueName: \"kubernetes.io/projected/b8d80d2a-2842-4360-bf92-87a9a905cfe3-kube-api-access-q48z6\") pod \"payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") " pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.281117 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.281016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d80d2a-2842-4360-bf92-87a9a905cfe3-plugins-config-volume\") pod \"payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") " pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.382256 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.382220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q48z6\" (UniqueName: \"kubernetes.io/projected/b8d80d2a-2842-4360-bf92-87a9a905cfe3-kube-api-access-q48z6\") pod \"payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") " pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.382486 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.382288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d80d2a-2842-4360-bf92-87a9a905cfe3-plugins-config-volume\") pod \"payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") " pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.383042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.383017    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d80d2a-2842-4360-bf92-87a9a905cfe3-plugins-config-volume\") pod \"payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") " pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.389826 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.389798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q48z6\" (UniqueName: \"kubernetes.io/projected/b8d80d2a-2842-4360-bf92-87a9a905cfe3-kube-api-access-q48z6\") pod \"payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") " pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.554576 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.554478    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:15.666763 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.666730    2563 generic.go:356] "Generic (PLEG): container finished" podID="34242533-f09d-4ada-9b53-6ee637edd6f5" containerID="cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4" exitCode=137
Sep 10 22:19:15.667142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.666780    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7617-572958"
Sep 10 22:19:15.667142 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.666835    2563 scope.go:117] "RemoveContainer" containerID="cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4"
Sep 10 22:19:15.675053 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.675035    2563 scope.go:117] "RemoveContainer" containerID="cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4"
Sep 10 22:19:15.675302 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:15.675286    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4\": container with ID starting with cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4 not found: ID does not exist" containerID="cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4"
Sep 10 22:19:15.675344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.675313    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4"} err="failed to get container status \"cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4\": rpc error: code = NotFound desc = could not find container \"cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4\": container with ID starting with cf32c3eb2862ba826768f814f5e93acf77f8e69e3d5efae12fdd76d7adcb00a4 not found: ID does not exist"
Sep 10 22:19:15.842767 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:15.842741    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"]
Sep 10 22:19:15.845403 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:15.845370    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8d80d2a_2842_4360_bf92_87a9a905cfe3.slice/crio-3327d2279414cb087541355c5db4c94cb14e25cc7463ca56361aa2cf6410ab51 WatchSource:0}: Error finding container 3327d2279414cb087541355c5db4c94cb14e25cc7463ca56361aa2cf6410ab51: Status 404 returned error can't find the container with id 3327d2279414cb087541355c5db4c94cb14e25cc7463ca56361aa2cf6410ab51
Sep 10 22:19:16.673509 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:16.673475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x" event={"ID":"b8d80d2a-2842-4360-bf92-87a9a905cfe3","Type":"ContainerStarted","Data":"5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602"}
Sep 10 22:19:16.673986 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:16.673537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x" event={"ID":"b8d80d2a-2842-4360-bf92-87a9a905cfe3","Type":"ContainerStarted","Data":"3327d2279414cb087541355c5db4c94cb14e25cc7463ca56361aa2cf6410ab51"}
Sep 10 22:19:16.673986 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:16.673772    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:16.692717 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:16.692669    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x" podStartSLOduration=1.692654925 podStartE2EDuration="1.692654925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:16.690907249 +0000 UTC m=+826.075412684" watchObservedRunningTime="2026-09-10 22:19:16.692654925 +0000 UTC m=+826.077160349"
Sep 10 22:19:16.697006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:16.696981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:19:17.613009 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.612938    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"]
Sep 10 22:19:17.616423 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.616405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:17.619406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.619385    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-44dc54-dockercfg-vsb4b\""
Sep 10 22:19:17.619746 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.619731    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-44dc54\""
Sep 10 22:19:17.627343 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.627315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"]
Sep 10 22:19:17.703192 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.703160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d342270c-0502-46ca-8d91-91785e9feecc-plugins-config-volume\") pod \"payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") " pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:17.703565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.703276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqhn9\" (UniqueName: \"kubernetes.io/projected/d342270c-0502-46ca-8d91-91785e9feecc-kube-api-access-hqhn9\") pod \"payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") " pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:17.804782 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.804743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d342270c-0502-46ca-8d91-91785e9feecc-plugins-config-volume\") pod \"payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") " pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:17.804950 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.804790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqhn9\" (UniqueName: \"kubernetes.io/projected/d342270c-0502-46ca-8d91-91785e9feecc-kube-api-access-hqhn9\") pod \"payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") " pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:17.805504 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.805483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d342270c-0502-46ca-8d91-91785e9feecc-plugins-config-volume\") pod \"payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") " pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:17.813250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.813223    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqhn9\" (UniqueName: \"kubernetes.io/projected/d342270c-0502-46ca-8d91-91785e9feecc-kube-api-access-hqhn9\") pod \"payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") " pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:17.928471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:17.928356    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:18.205663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:18.205634    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"]
Sep 10 22:19:18.207373 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:18.207339    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd342270c_0502_46ca_8d91_91785e9feecc.slice/crio-21c772b3ee61c41262d5f81a9172c3cc72f8e4365e7797ed4567fe3dd696049d WatchSource:0}: Error finding container 21c772b3ee61c41262d5f81a9172c3cc72f8e4365e7797ed4567fe3dd696049d: Status 404 returned error can't find the container with id 21c772b3ee61c41262d5f81a9172c3cc72f8e4365e7797ed4567fe3dd696049d
Sep 10 22:19:18.686769 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:18.686730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b" event={"ID":"d342270c-0502-46ca-8d91-91785e9feecc","Type":"ContainerStarted","Data":"c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d"}
Sep 10 22:19:18.686966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:18.686806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b" event={"ID":"d342270c-0502-46ca-8d91-91785e9feecc","Type":"ContainerStarted","Data":"21c772b3ee61c41262d5f81a9172c3cc72f8e4365e7797ed4567fe3dd696049d"}
Sep 10 22:19:18.686966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:18.686889    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:18.703112 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:18.703062    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b" podStartSLOduration=1.703044762 podStartE2EDuration="1.703044762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:18.701053232 +0000 UTC m=+828.085558658" watchObservedRunningTime="2026-09-10 22:19:18.703044762 +0000 UTC m=+828.087550188"
Sep 10 22:19:20.593066 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.592993    2563 patch_prober.go:28] interesting pod/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 10 22:19:20.593066 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.593048    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 10 22:19:20.698062 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.698031    2563 generic.go:356] "Generic (PLEG): container finished" podID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerID="9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12" exitCode=0
Sep 10 22:19:20.698228 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.698106    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" event={"ID":"56487366-5ec6-41d9-afd0-9534f65f1b3b","Type":"ContainerDied","Data":"9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12"}
Sep 10 22:19:20.698546 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.698531    2563 scope.go:117] "RemoveContainer" containerID="9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12"
Sep 10 22:19:20.699572 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.699548    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:20.699623 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.699614    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:20.699661 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:20.699646    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:21.629179 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:21.629135    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:19:21.705454 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:21.705408    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" event={"ID":"56487366-5ec6-41d9-afd0-9534f65f1b3b","Type":"ContainerStarted","Data":"3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8"}
Sep 10 22:19:22.465579 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:22.465545    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:22.465785 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:22.465764    2563 patch_prober.go:28] interesting pod/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 10 22:19:22.465874 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:22.465814    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 10 22:19:23.465517 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:23.465481    2563 patch_prober.go:28] interesting pod/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 10 22:19:23.465876 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:23.465561    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 10 22:19:24.465711 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.465675    2563 patch_prober.go:28] interesting pod/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 10 22:19:24.466094 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.465746    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 10 22:19:24.590273 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.590234    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"]
Sep 10 22:19:24.594223 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.594198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.596242 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.596220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-7660dbe4-kserve-self-signed-certs\""
Sep 10 22:19:24.603773 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.603748    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"]
Sep 10 22:19:24.671182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.671148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-home\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.671182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.671181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-dshm\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.671377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.671211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tmp-dir\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.671377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.671288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lsqw\" (UniqueName: \"kubernetes.io/projected/df6ff56a-8b6d-4553-84ed-d783d2e837b8-kube-api-access-9lsqw\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.671377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.671319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tls-certs\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.671377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.671333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-model-cache\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.772237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.772206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-home\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.772237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.772239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-dshm\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.772497 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.772260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tmp-dir\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.772497 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.772299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lsqw\" (UniqueName: \"kubernetes.io/projected/df6ff56a-8b6d-4553-84ed-d783d2e837b8-kube-api-access-9lsqw\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.772497 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.772343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tls-certs\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.772497 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.772383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-model-cache\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.774328 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.774303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-home\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.775368 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.775346    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tmp-dir\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.775470 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.775432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-model-cache\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.775930 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.775913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-dshm\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.776796 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.776768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tls-certs\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.780197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.780180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lsqw\" (UniqueName: \"kubernetes.io/projected/df6ff56a-8b6d-4553-84ed-d783d2e837b8-kube-api-access-9lsqw\") pod \"e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") " pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:24.903708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:24.903673    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:25.200652 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:25.200573    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"]
Sep 10 22:19:25.204100 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:25.204075    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf6ff56a_8b6d_4553_84ed_d783d2e837b8.slice/crio-a853593eab2644bd85e9a2bb8cbb9f260f83bc776a43158f57496f50c512dbd6 WatchSource:0}: Error finding container a853593eab2644bd85e9a2bb8cbb9f260f83bc776a43158f57496f50c512dbd6: Status 404 returned error can't find the container with id a853593eab2644bd85e9a2bb8cbb9f260f83bc776a43158f57496f50c512dbd6
Sep 10 22:19:25.465697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:25.465617    2563 patch_prober.go:28] interesting pod/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 10 22:19:25.465914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:25.465891    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 10 22:19:25.735690 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:25.735655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6" event={"ID":"df6ff56a-8b6d-4553-84ed-d783d2e837b8","Type":"ContainerStarted","Data":"2f9ce69d9a836609205e068efdfab9e8e43fe7a0756b24fe6f03cc5945fc4a11"}
Sep 10 22:19:25.735963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:25.735940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6" event={"ID":"df6ff56a-8b6d-4553-84ed-d783d2e837b8","Type":"ContainerStarted","Data":"a853593eab2644bd85e9a2bb8cbb9f260f83bc776a43158f57496f50c512dbd6"}
Sep 10 22:19:25.753678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:25.753617    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6" podStartSLOduration=1.753594477 podStartE2EDuration="1.753594477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:25.751100414 +0000 UTC m=+835.135605839" watchObservedRunningTime="2026-09-10 22:19:25.753594477 +0000 UTC m=+835.138099906"
Sep 10 22:19:26.464997 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:26.464960    2563 patch_prober.go:28] interesting pod/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Sep 10 22:19:26.465667 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:26.465250    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Sep 10 22:19:27.469688 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:27.469658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:27.470070 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:27.469991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:27.470718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:27.470697    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:19:27.681959 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:27.681918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:29.694414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:29.694382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:19:33.313232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:33.313196    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:19:34.795728 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.795645    2563 generic.go:356] "Generic (PLEG): container finished" podID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerID="ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31" exitCode=0
Sep 10 22:19:34.795728 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.795698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" event={"ID":"129e09c3-9ae9-4c39-9493-ea5a2654b5aa","Type":"ContainerDied","Data":"ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31"}
Sep 10 22:19:34.796143 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.796085    2563 scope.go:117] "RemoveContainer" containerID="ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31"
Sep 10 22:19:34.797297 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.797266    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:34.797400 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.797339    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:34.797400 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.797391    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:34.904186 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.904149    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:34.904378 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.904218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:34.918522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:34.918489    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:35.802411 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:35.802375    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" event={"ID":"129e09c3-9ae9-4c39-9493-ea5a2654b5aa","Type":"ContainerStarted","Data":"9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca"}
Sep 10 22:19:35.813698 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:35.813676    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:19:36.370976 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:36.370940    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:36.371255 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:36.371235    2563 patch_prober.go:28] interesting pod/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 22:19:36.371312 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:36.371293    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 22:19:37.371596 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:37.371563    2563 patch_prober.go:28] interesting pod/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 22:19:37.371990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:37.371619    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 22:19:38.371952 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.371914    2563 patch_prober.go:28] interesting pod/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 22:19:38.372348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.371972    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 22:19:38.691213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.691133    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"]
Sep 10 22:19:38.693557 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.693541    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"
Sep 10 22:19:38.695854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.695828    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 22:19:38.695854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.695835    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlfhj\""
Sep 10 22:19:38.701765 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.701739    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"]
Sep 10 22:19:38.811237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.811196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24a0faa2-3df6-441c-b005-a470453b8c81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl\" (UID: \"24a0faa2-3df6-441c-b005-a470453b8c81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"
Sep 10 22:19:38.912807 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.912760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24a0faa2-3df6-441c-b005-a470453b8c81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl\" (UID: \"24a0faa2-3df6-441c-b005-a470453b8c81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"
Sep 10 22:19:38.913896 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:38.913868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24a0faa2-3df6-441c-b005-a470453b8c81-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl\" (UID: \"24a0faa2-3df6-441c-b005-a470453b8c81\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"
Sep 10 22:19:39.005426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:39.005398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"
Sep 10 22:19:39.294100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:39.294074    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"]
Sep 10 22:19:39.296362 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:39.296335    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a0faa2_3df6_441c_b005_a470453b8c81.slice/crio-72056e3d66c5ad057250621b05129f03862cf23082415136a115257119a8c16b WatchSource:0}: Error finding container 72056e3d66c5ad057250621b05129f03862cf23082415136a115257119a8c16b: Status 404 returned error can't find the container with id 72056e3d66c5ad057250621b05129f03862cf23082415136a115257119a8c16b
Sep 10 22:19:39.371587 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:39.371551    2563 patch_prober.go:28] interesting pod/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 22:19:39.371759 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:39.371611    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 22:19:39.842419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:39.842383    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl" event={"ID":"24a0faa2-3df6-441c-b005-a470453b8c81","Type":"ContainerStarted","Data":"72056e3d66c5ad057250621b05129f03862cf23082415136a115257119a8c16b"}
Sep 10 22:19:40.371129 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:40.371096    2563 patch_prober.go:28] interesting pod/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 22:19:40.371337 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:40.371150    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 22:19:41.371995 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:41.371953    2563 patch_prober.go:28] interesting pod/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 22:19:41.372495 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:41.372025    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 22:19:42.375367 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:42.375330    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:42.375735 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:42.375698    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:42.376361 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:42.376322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:19:42.860654 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:42.860612    2563 generic.go:356] "Generic (PLEG): container finished" podID="24a0faa2-3df6-441c-b005-a470453b8c81" containerID="6adf7ae176ee4554218880d5a59e70bcdf4490f4eddce0ba926071d8e00bcbc7" exitCode=0
Sep 10 22:19:42.860811 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:42.860697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl" event={"ID":"24a0faa2-3df6-441c-b005-a470453b8c81","Type":"ContainerDied","Data":"6adf7ae176ee4554218880d5a59e70bcdf4490f4eddce0ba926071d8e00bcbc7"}
Sep 10 22:19:44.002178 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:44.002155    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"
Sep 10 22:19:44.176544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:44.176424    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24a0faa2-3df6-441c-b005-a470453b8c81-maas-api-service-ca\") pod \"24a0faa2-3df6-441c-b005-a470453b8c81\" (UID: \"24a0faa2-3df6-441c-b005-a470453b8c81\") "
Sep 10 22:19:44.176801 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:44.176770    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24a0faa2-3df6-441c-b005-a470453b8c81-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "24a0faa2-3df6-441c-b005-a470453b8c81" (UID: "24a0faa2-3df6-441c-b005-a470453b8c81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:19:44.278489 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:44.278431    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/24a0faa2-3df6-441c-b005-a470453b8c81-maas-api-service-ca\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:19:44.872628 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:44.872592    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl" event={"ID":"24a0faa2-3df6-441c-b005-a470453b8c81","Type":"ContainerDied","Data":"72056e3d66c5ad057250621b05129f03862cf23082415136a115257119a8c16b"}
Sep 10 22:19:44.872793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:44.872649    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72056e3d66c5ad057250621b05129f03862cf23082415136a115257119a8c16b"
Sep 10 22:19:44.872793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:44.872607    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"
Sep 10 22:19:50.404944 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:50.404898    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:19:55.286413 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.286379    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"]
Sep 10 22:19:55.286868 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.286623    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x" podUID="b8d80d2a-2842-4360-bf92-87a9a905cfe3" containerName="payload-processing" containerID="cri-o://5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602" gracePeriod=30
Sep 10 22:19:55.453317 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.453285    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"]
Sep 10 22:19:55.455726 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.455696    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-938f94d8-4dfwl"]
Sep 10 22:19:55.539219 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.539150    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:55.577486 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.577430    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d80d2a-2842-4360-bf92-87a9a905cfe3-plugins-config-volume\") pod \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") "
Sep 10 22:19:55.577647 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.577565    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q48z6\" (UniqueName: \"kubernetes.io/projected/b8d80d2a-2842-4360-bf92-87a9a905cfe3-kube-api-access-q48z6\") pod \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\" (UID: \"b8d80d2a-2842-4360-bf92-87a9a905cfe3\") "
Sep 10 22:19:55.577791 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.577769    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8d80d2a-2842-4360-bf92-87a9a905cfe3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8d80d2a-2842-4360-bf92-87a9a905cfe3" (UID: "b8d80d2a-2842-4360-bf92-87a9a905cfe3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:19:55.577906 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.577891    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8d80d2a-2842-4360-bf92-87a9a905cfe3-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:19:55.579843 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.579810    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d80d2a-2842-4360-bf92-87a9a905cfe3-kube-api-access-q48z6" (OuterVolumeSpecName: "kube-api-access-q48z6") pod "b8d80d2a-2842-4360-bf92-87a9a905cfe3" (UID: "b8d80d2a-2842-4360-bf92-87a9a905cfe3"). InnerVolumeSpecName "kube-api-access-q48z6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:19:55.679087 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.679053    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q48z6\" (UniqueName: \"kubernetes.io/projected/b8d80d2a-2842-4360-bf92-87a9a905cfe3-kube-api-access-q48z6\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:19:55.930350 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.930261    2563 generic.go:356] "Generic (PLEG): container finished" podID="b8d80d2a-2842-4360-bf92-87a9a905cfe3" containerID="5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602" exitCode=0
Sep 10 22:19:55.930350 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.930312    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"
Sep 10 22:19:55.930608 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.930342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x" event={"ID":"b8d80d2a-2842-4360-bf92-87a9a905cfe3","Type":"ContainerDied","Data":"5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602"}
Sep 10 22:19:55.930608 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.930399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x" event={"ID":"b8d80d2a-2842-4360-bf92-87a9a905cfe3","Type":"ContainerDied","Data":"3327d2279414cb087541355c5db4c94cb14e25cc7463ca56361aa2cf6410ab51"}
Sep 10 22:19:55.930608 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.930422    2563 scope.go:117] "RemoveContainer" containerID="5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602"
Sep 10 22:19:55.939649 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.939628    2563 scope.go:117] "RemoveContainer" containerID="5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602"
Sep 10 22:19:55.939965 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:19:55.939943    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602\": container with ID starting with 5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602 not found: ID does not exist" containerID="5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602"
Sep 10 22:19:55.940039 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.939978    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602"} err="failed to get container status \"5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602\": rpc error: code = NotFound desc = could not find container \"5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602\": container with ID starting with 5658c0b1800069869c63cf4e2491eb7fd113b0a1268ba832bb7bb20d8c18a602 not found: ID does not exist"
Sep 10 22:19:55.950627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.950586    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"]
Sep 10 22:19:55.952630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:55.952611    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-938f94d8-66c5477cdf-xcx4x"]
Sep 10 22:19:56.509573 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.509523    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"]
Sep 10 22:19:56.510100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.510079    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d80d2a-2842-4360-bf92-87a9a905cfe3" containerName="payload-processing"
Sep 10 22:19:56.510192 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.510104    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d80d2a-2842-4360-bf92-87a9a905cfe3" containerName="payload-processing"
Sep 10 22:19:56.510192 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.510152    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a0faa2-3df6-441c-b005-a470453b8c81" containerName="revoke-keys"
Sep 10 22:19:56.510192 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.510161    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a0faa2-3df6-441c-b005-a470453b8c81" containerName="revoke-keys"
Sep 10 22:19:56.510402 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.510262    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8d80d2a-2842-4360-bf92-87a9a905cfe3" containerName="payload-processing"
Sep 10 22:19:56.510402 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.510278    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a0faa2-3df6-441c-b005-a470453b8c81" containerName="revoke-keys"
Sep 10 22:19:56.513274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.513250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.515610 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.515591    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-7178ee60-gw-openshift-default-dockercfg-zqsnq\""
Sep 10 22:19:56.525176 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.525157    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"]
Sep 10 22:19:56.594366 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-envoy\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594656 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594637    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istiod-ca-cert\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594734 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-data\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594734 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-certs\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-podinfo\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-credential-socket\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594806    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-token\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594920 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-socket\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.594961 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.594919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tg5j\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-kube-api-access-4tg5j\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.695970 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.695933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-socket\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.695970 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.695969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tg5j\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-kube-api-access-4tg5j\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-envoy\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istiod-ca-cert\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-data\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-certs\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-podinfo\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696126    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-credential-socket\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-token\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696473    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-socket\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-credential-socket\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696760 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-certs\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.696760 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.696738    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-data\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.697082 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.697018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istiod-ca-cert\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.698565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.698541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-envoy\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.698774 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.698755    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-podinfo\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.704122 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.704095    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-token\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.704757 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.704734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tg5j\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-kube-api-access-4tg5j\") pod \"e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") " pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:56.824711 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:56.824642    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:57.099130 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.099077    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"]
Sep 10 22:19:57.101405 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:19:57.101379    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod126ae161_5720_4cd6_8f35_74cf7fae7a5a.slice/crio-b853942898518eb9c12d6b017bf9760241e2e41644f98e65699e9dcf0148fd5e WatchSource:0}: Error finding container b853942898518eb9c12d6b017bf9760241e2e41644f98e65699e9dcf0148fd5e: Status 404 returned error can't find the container with id b853942898518eb9c12d6b017bf9760241e2e41644f98e65699e9dcf0148fd5e
Sep 10 22:19:57.103485 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.103423    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:57.103554 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.103523    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:57.103554 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.103551    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:19:57.110956 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.110932    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24a0faa2-3df6-441c-b005-a470453b8c81" path="/var/lib/kubelet/pods/24a0faa2-3df6-441c-b005-a470453b8c81/volumes"
Sep 10 22:19:57.111271 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.111259    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d80d2a-2842-4360-bf92-87a9a905cfe3" path="/var/lib/kubelet/pods/b8d80d2a-2842-4360-bf92-87a9a905cfe3/volumes"
Sep 10 22:19:57.943817 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.943777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" event={"ID":"126ae161-5720-4cd6-8f35-74cf7fae7a5a","Type":"ContainerStarted","Data":"cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02"}
Sep 10 22:19:57.944289 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.943840    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" event={"ID":"126ae161-5720-4cd6-8f35-74cf7fae7a5a","Type":"ContainerStarted","Data":"b853942898518eb9c12d6b017bf9760241e2e41644f98e65699e9dcf0148fd5e"}
Sep 10 22:19:57.962709 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:57.962645    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" podStartSLOduration=1.962624415 podStartE2EDuration="1.962624415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:19:57.960357293 +0000 UTC m=+867.344862742" watchObservedRunningTime="2026-09-10 22:19:57.962624415 +0000 UTC m=+867.347129843"
Sep 10 22:19:58.824954 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:58.824921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:58.829885 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:58.829858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:58.948869 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:58.948845    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:19:58.949789 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:19:58.949769    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:20:00.428741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.428709    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"]
Sep 10 22:20:00.431609 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.431584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:00.433858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.433834    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-7178ee60\""
Sep 10 22:20:00.433976 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.433895    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-7178ee60-dockercfg-4ph2x\""
Sep 10 22:20:00.444108 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.444086    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"]
Sep 10 22:20:00.532106 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.532074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bwgd\" (UniqueName: \"kubernetes.io/projected/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-kube-api-access-5bwgd\") pod \"payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") " pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:00.532298 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.532123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") " pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:00.633880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.633832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bwgd\" (UniqueName: \"kubernetes.io/projected/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-kube-api-access-5bwgd\") pod \"payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") " pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:00.634047 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.633921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") " pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:00.634632 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.634613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-plugins-config-volume\") pod \"payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") " pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:00.642760 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.642737    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bwgd\" (UniqueName: \"kubernetes.io/projected/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-kube-api-access-5bwgd\") pod \"payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") " pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:00.746210 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:00.746170    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:01.018883 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:01.018852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"]
Sep 10 22:20:01.021207 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:20:01.021163    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f37ec0_d5c0_4bb0_89b8_762bc047f5e0.slice/crio-8d057b3860e3992a4e8dd7ead2000fdff7dae2776e06a4875820b96939d26375 WatchSource:0}: Error finding container 8d057b3860e3992a4e8dd7ead2000fdff7dae2776e06a4875820b96939d26375: Status 404 returned error can't find the container with id 8d057b3860e3992a4e8dd7ead2000fdff7dae2776e06a4875820b96939d26375
Sep 10 22:20:01.974885 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:01.974851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j" event={"ID":"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0","Type":"ContainerStarted","Data":"da808ea29303630d89bd990362fa454f391858d497e56fa96cdf8ada002fc099"}
Sep 10 22:20:01.975250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:01.974900    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j" event={"ID":"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0","Type":"ContainerStarted","Data":"8d057b3860e3992a4e8dd7ead2000fdff7dae2776e06a4875820b96939d26375"}
Sep 10 22:20:01.975250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:01.974979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:02.008557 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:02.008510    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j" podStartSLOduration=2.008495324 podStartE2EDuration="2.008495324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:02.007551378 +0000 UTC m=+871.392056803" watchObservedRunningTime="2026-09-10 22:20:02.008495324 +0000 UTC m=+871.393000783"
Sep 10 22:20:08.307511 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:08.307481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:20:12.982631 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:12.982598    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:19.081137 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:19.081101    2563 generic.go:356] "Generic (PLEG): container finished" podID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerID="cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02" exitCode=0
Sep 10 22:20:19.081567 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:19.081173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" event={"ID":"126ae161-5720-4cd6-8f35-74cf7fae7a5a","Type":"ContainerDied","Data":"cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02"}
Sep 10 22:20:19.081715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:19.081698    2563 scope.go:117] "RemoveContainer" containerID="cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02"
Sep 10 22:20:19.082791 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:19.082766    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:20:19.082850 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:19.082836    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:20:19.082889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:19.082871    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:20:20.088348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:20.088311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" event={"ID":"126ae161-5720-4cd6-8f35-74cf7fae7a5a","Type":"ContainerStarted","Data":"4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f"}
Sep 10 22:20:20.825862 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:20.825821    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:20:20.826308 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:20.826281    2563 patch_prober.go:28] interesting pod/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 22:20:20.826526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:20.826498    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 22:20:21.825314 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:21.825271    2563 patch_prober.go:28] interesting pod/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 22:20:21.825822 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:21.825338    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 22:20:22.825236 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:22.825195    2563 patch_prober.go:28] interesting pod/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 22:20:22.825572 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:22.825541    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 22:20:23.825519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:23.825475    2563 patch_prober.go:28] interesting pod/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 22:20:23.825833 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:23.825547    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 22:20:24.825934 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:24.825890    2563 patch_prober.go:28] interesting pod/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 22:20:24.826494 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:24.825967    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 22:20:25.829645 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:25.829618    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:20:25.830102 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:25.829901    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:20:25.830635 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:25.830620    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:20:30.308139 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:30.308101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:20:31.023013 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.022981    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"]
Sep 10 22:20:31.023289 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.023242    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j" podUID="18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" containerName="payload-processing" containerID="cri-o://da808ea29303630d89bd990362fa454f391858d497e56fa96cdf8ada002fc099" gracePeriod=30
Sep 10 22:20:31.141071 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.141029    2563 generic.go:356] "Generic (PLEG): container finished" podID="18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" containerID="da808ea29303630d89bd990362fa454f391858d497e56fa96cdf8ada002fc099" exitCode=0
Sep 10 22:20:31.141181 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.141096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j" event={"ID":"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0","Type":"ContainerDied","Data":"da808ea29303630d89bd990362fa454f391858d497e56fa96cdf8ada002fc099"}
Sep 10 22:20:31.284461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.284378    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:31.331703 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.331674    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-plugins-config-volume\") pod \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") "
Sep 10 22:20:31.332205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.331805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bwgd\" (UniqueName: \"kubernetes.io/projected/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-kube-api-access-5bwgd\") pod \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\" (UID: \"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0\") "
Sep 10 22:20:31.332205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.332122    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" (UID: "18f37ec0-d5c0-4bb0-89b8-762bc047f5e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:20:31.334743 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.334716    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"]
Sep 10 22:20:31.334986 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.334959    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy" containerID="cri-o://4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f" gracePeriod=30
Sep 10 22:20:31.335776 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.335740    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-kube-api-access-5bwgd" (OuterVolumeSpecName: "kube-api-access-5bwgd") pod "18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" (UID: "18f37ec0-d5c0-4bb0-89b8-762bc047f5e0"). InnerVolumeSpecName "kube-api-access-5bwgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:20:31.432876 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.432841    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:31.432876 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:31.432873    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bwgd\" (UniqueName: \"kubernetes.io/projected/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0-kube-api-access-5bwgd\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:32.174458 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:32.171349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j" event={"ID":"18f37ec0-d5c0-4bb0-89b8-762bc047f5e0","Type":"ContainerDied","Data":"8d057b3860e3992a4e8dd7ead2000fdff7dae2776e06a4875820b96939d26375"}
Sep 10 22:20:32.174458 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:32.171417    2563 scope.go:117] "RemoveContainer" containerID="da808ea29303630d89bd990362fa454f391858d497e56fa96cdf8ada002fc099"
Sep 10 22:20:32.174458 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:32.171598    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"
Sep 10 22:20:32.197960 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:32.197932    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"]
Sep 10 22:20:32.203934 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:32.203905    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-7178ee60-6c76bdd7f4-s9p2j"]
Sep 10 22:20:33.114901 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:33.114863    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" path="/var/lib/kubelet/pods/18f37ec0-d5c0-4bb0-89b8-762bc047f5e0/volumes"
Sep 10 22:20:36.413063 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.413026    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"]
Sep 10 22:20:36.413393 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.413386    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" containerName="payload-processing"
Sep 10 22:20:36.413431 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.413397    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" containerName="payload-processing"
Sep 10 22:20:36.413499 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.413493    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f37ec0-d5c0-4bb0-89b8-762bc047f5e0" containerName="payload-processing"
Sep 10 22:20:36.415733 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.415717    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.417829 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.417807    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-178c50fc\""
Sep 10 22:20:36.425660 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.425637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"]
Sep 10 22:20:36.438614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.438584    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"]
Sep 10 22:20:36.441485 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.441466    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.444652 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.444628    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-178c50fc-dockercfg-mbjpt\""
Sep 10 22:20:36.454250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.454224    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"]
Sep 10 22:20:36.476331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.476291    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308cb5ab-547d-4334-aa45-697fb6420197-plugins-config-volume\") pod \"payload-processing-e2e-ait-178c50fc-7f554495b9-sr548\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") " pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.476331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.476325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6098c1-b038-4b69-ab96-60ab5da82bf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.476599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.476405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgd9v\" (UniqueName: \"kubernetes.io/projected/0a6098c1-b038-4b69-ab96-60ab5da82bf0-kube-api-access-jgd9v\") pod \"payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.476599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.476466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm8hj\" (UniqueName: \"kubernetes.io/projected/308cb5ab-547d-4334-aa45-697fb6420197-kube-api-access-mm8hj\") pod \"payload-processing-e2e-ait-178c50fc-7f554495b9-sr548\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") " pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.577278 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.577253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308cb5ab-547d-4334-aa45-697fb6420197-plugins-config-volume\") pod \"payload-processing-e2e-ait-178c50fc-7f554495b9-sr548\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") " pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.577364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.577284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6098c1-b038-4b69-ab96-60ab5da82bf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.577364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.577337    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgd9v\" (UniqueName: \"kubernetes.io/projected/0a6098c1-b038-4b69-ab96-60ab5da82bf0-kube-api-access-jgd9v\") pod \"payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.577483 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.577367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm8hj\" (UniqueName: \"kubernetes.io/projected/308cb5ab-547d-4334-aa45-697fb6420197-kube-api-access-mm8hj\") pod \"payload-processing-e2e-ait-178c50fc-7f554495b9-sr548\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") " pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.578020 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.577995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6098c1-b038-4b69-ab96-60ab5da82bf0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.578167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.578143    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308cb5ab-547d-4334-aa45-697fb6420197-plugins-config-volume\") pod \"payload-processing-e2e-ait-178c50fc-7f554495b9-sr548\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") " pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.585012 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.584989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm8hj\" (UniqueName: \"kubernetes.io/projected/308cb5ab-547d-4334-aa45-697fb6420197-kube-api-access-mm8hj\") pod \"payload-processing-e2e-ait-178c50fc-7f554495b9-sr548\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") " pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.585184 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.585161    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgd9v\" (UniqueName: \"kubernetes.io/projected/0a6098c1-b038-4b69-ab96-60ab5da82bf0-kube-api-access-jgd9v\") pod \"payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.601448 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.601384    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:20:36.678414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678381    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-envoy\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678414    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-data\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678429    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-credential-socket\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678493    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tg5j\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-kube-api-access-4tg5j\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678527    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-socket\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678580    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-podinfo\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678848 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678679    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-token\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678848 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678716    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:20:36.678848 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istiod-ca-cert\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.678848 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678734    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:20:36.679054 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678858    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-certs\") pod \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\" (UID: \"126ae161-5720-4cd6-8f35-74cf7fae7a5a\") "
Sep 10 22:20:36.679054 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.678871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:20:36.679159 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.679051    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:20:36.679214 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.679151    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:20:36.679537 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.679518    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.679597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.679544    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.679597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.679560    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.679597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.679573    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.679597 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.679586    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.680847 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.680816    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:20:36.680958 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.680906    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-kube-api-access-4tg5j" (OuterVolumeSpecName: "kube-api-access-4tg5j") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "kube-api-access-4tg5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:20:36.680958 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.680944    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:20:36.681036 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.680950    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "126ae161-5720-4cd6-8f35-74cf7fae7a5a" (UID: "126ae161-5720-4cd6-8f35-74cf7fae7a5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:20:36.726132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.726101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:36.753366 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.753338    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:36.780562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.780537    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.780670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.780568    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tg5j\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-kube-api-access-4tg5j\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.780670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.780584    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.780670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.780599    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/126ae161-5720-4cd6-8f35-74cf7fae7a5a-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:20:36.999838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:36.999808    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"]
Sep 10 22:20:37.001552 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:20:37.001528    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6098c1_b038_4b69_ab96_60ab5da82bf0.slice/crio-e02ed4131b97b06510ec4c66b8b6c5aa85925a8f67d06ec4ebdca10596b13314 WatchSource:0}: Error finding container e02ed4131b97b06510ec4c66b8b6c5aa85925a8f67d06ec4ebdca10596b13314: Status 404 returned error can't find the container with id e02ed4131b97b06510ec4c66b8b6c5aa85925a8f67d06ec4ebdca10596b13314
Sep 10 22:20:37.031261 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.031215    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"]
Sep 10 22:20:37.033182 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:20:37.033159    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308cb5ab_547d_4334_aa45_697fb6420197.slice/crio-89ac268a9b6e4c033e8a63d3ae834ddedba2c154fca38437f1b7d84d0b4dce20 WatchSource:0}: Error finding container 89ac268a9b6e4c033e8a63d3ae834ddedba2c154fca38437f1b7d84d0b4dce20: Status 404 returned error can't find the container with id 89ac268a9b6e4c033e8a63d3ae834ddedba2c154fca38437f1b7d84d0b4dce20
Sep 10 22:20:37.203566 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.203532    2563 generic.go:356] "Generic (PLEG): container finished" podID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerID="4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f" exitCode=0
Sep 10 22:20:37.203751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.203609    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"
Sep 10 22:20:37.203751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.203614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" event={"ID":"126ae161-5720-4cd6-8f35-74cf7fae7a5a","Type":"ContainerDied","Data":"4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f"}
Sep 10 22:20:37.203751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.203694    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm" event={"ID":"126ae161-5720-4cd6-8f35-74cf7fae7a5a","Type":"ContainerDied","Data":"b853942898518eb9c12d6b017bf9760241e2e41644f98e65699e9dcf0148fd5e"}
Sep 10 22:20:37.203751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.203727    2563 scope.go:117] "RemoveContainer" containerID="4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f"
Sep 10 22:20:37.205774 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.205608    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f" event={"ID":"0a6098c1-b038-4b69-ab96-60ab5da82bf0","Type":"ContainerStarted","Data":"f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689"}
Sep 10 22:20:37.205774 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.205653    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f" event={"ID":"0a6098c1-b038-4b69-ab96-60ab5da82bf0","Type":"ContainerStarted","Data":"e02ed4131b97b06510ec4c66b8b6c5aa85925a8f67d06ec4ebdca10596b13314"}
Sep 10 22:20:37.205774 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.205694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:37.207549 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.207525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548" event={"ID":"308cb5ab-547d-4334-aa45-697fb6420197","Type":"ContainerStarted","Data":"104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb"}
Sep 10 22:20:37.207669 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.207560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548" event={"ID":"308cb5ab-547d-4334-aa45-697fb6420197","Type":"ContainerStarted","Data":"89ac268a9b6e4c033e8a63d3ae834ddedba2c154fca38437f1b7d84d0b4dce20"}
Sep 10 22:20:37.207669 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.207600    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:37.212872 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.212854    2563 scope.go:117] "RemoveContainer" containerID="cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02"
Sep 10 22:20:37.221074 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.221026    2563 scope.go:117] "RemoveContainer" containerID="4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f"
Sep 10 22:20:37.221395 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:20:37.221365    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f\": container with ID starting with 4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f not found: ID does not exist" containerID="4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f"
Sep 10 22:20:37.221537 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.221515    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f"} err="failed to get container status \"4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f\": rpc error: code = NotFound desc = could not find container \"4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f\": container with ID starting with 4f6acb6a5e4441ae3b7ea8238257c5f2ba713adc991a3b4e13199326d462c71f not found: ID does not exist"
Sep 10 22:20:37.221632 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.221613    2563 scope.go:117] "RemoveContainer" containerID="cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02"
Sep 10 22:20:37.222014 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:20:37.221986    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02\": container with ID starting with cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02 not found: ID does not exist" containerID="cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02"
Sep 10 22:20:37.222096 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.222023    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02"} err="failed to get container status \"cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02\": rpc error: code = NotFound desc = could not find container \"cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02\": container with ID starting with cd83063dca0c688a47863489ec38fc623764592b81a441c1534209b9e2d37e02 not found: ID does not exist"
Sep 10 22:20:37.223330 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.223285    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f" podStartSLOduration=1.223272829 podStartE2EDuration="1.223272829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:37.220833396 +0000 UTC m=+906.605338850" watchObservedRunningTime="2026-09-10 22:20:37.223272829 +0000 UTC m=+906.607778254"
Sep 10 22:20:37.234660 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.234638    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"]
Sep 10 22:20:37.240607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.240587    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-7178ee60-gw-openshift-default-54945f8d64-zf7cm"]
Sep 10 22:20:37.252880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:37.252803    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548" podStartSLOduration=1.252787571 podStartE2EDuration="1.252787571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:37.2511684 +0000 UTC m=+906.635673827" watchObservedRunningTime="2026-09-10 22:20:37.252787571 +0000 UTC m=+906.637292998"
Sep 10 22:20:39.111718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:39.111687    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" path="/var/lib/kubelet/pods/126ae161-5720-4cd6-8f35-74cf7fae7a5a/volumes"
Sep 10 22:20:48.212181 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:48.212148    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:20:48.214779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:48.214750    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:20:48.215086 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:48.215069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:20:53.786870 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.786839    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"]
Sep 10 22:20:53.787258 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.787188    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy"
Sep 10 22:20:53.787258 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.787198    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy"
Sep 10 22:20:53.787258 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.787209    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy"
Sep 10 22:20:53.787258 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.787215    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy"
Sep 10 22:20:53.787394 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.787277    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy"
Sep 10 22:20:53.787394 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.787287    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="126ae161-5720-4cd6-8f35-74cf7fae7a5a" containerName="istio-proxy"
Sep 10 22:20:53.790206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.790186    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:53.792385 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.792359    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-7660dbe4-kserve-self-signed-certs\""
Sep 10 22:20:53.801573 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.801547    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"]
Sep 10 22:20:53.930461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.930414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-dshm\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:53.930634 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.930510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-model-cache\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:53.930634 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.930556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-home\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:53.930634 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.930576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tmp-dir\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:53.930634 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.930607    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tls-certs\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:53.930825 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:53.930704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqk9s\" (UniqueName: \"kubernetes.io/projected/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-kube-api-access-tqk9s\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-dshm\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-model-cache\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032354 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-home\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032354 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tmp-dir\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032513 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tls-certs\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032513 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqk9s\" (UniqueName: \"kubernetes.io/projected/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-kube-api-access-tqk9s\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032617 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tmp-dir\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032672 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-model-cache\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.032708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.032671    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-home\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.034671 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.034649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-dshm\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.034853 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.034835    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tls-certs\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.039861 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.039817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqk9s\" (UniqueName: \"kubernetes.io/projected/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-kube-api-access-tqk9s\") pod \"e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") " pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.101139 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.101112    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:20:54.373687 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:54.373650    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"]
Sep 10 22:20:54.376484 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:20:54.376422    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc672ac99_0c4a_4c5c_bd42_d1192bc0f4cb.slice/crio-bf32bb6a56bef00144d760a263555dc90793114a387d432aef8fd349b403c4d2 WatchSource:0}: Error finding container bf32bb6a56bef00144d760a263555dc90793114a387d432aef8fd349b403c4d2: Status 404 returned error can't find the container with id bf32bb6a56bef00144d760a263555dc90793114a387d432aef8fd349b403c4d2
Sep 10 22:20:55.302433 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:55.302394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2" event={"ID":"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb","Type":"ContainerStarted","Data":"ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29"}
Sep 10 22:20:55.302828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:55.302467    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2" event={"ID":"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb","Type":"ContainerStarted","Data":"bf32bb6a56bef00144d760a263555dc90793114a387d432aef8fd349b403c4d2"}
Sep 10 22:20:55.316727 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:20:55.316683    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2" podStartSLOduration=2.316666348 podStartE2EDuration="2.316666348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:20:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:20:55.315284677 +0000 UTC m=+924.699790101" watchObservedRunningTime="2026-09-10 22:20:55.316666348 +0000 UTC m=+924.701171772"
Sep 10 22:21:03.301637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:03.301597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:21:04.101233 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:04.101196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:21:04.101403 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:04.101245    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:21:04.113900 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:04.113874    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:21:04.364498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:04.364392    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:21:05.702955 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.702923    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"]
Sep 10 22:21:05.705280 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.705263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.707487 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.707458    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-44dc54\"/\"openshift-service-ca.crt\""
Sep 10 22:21:05.708182 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.708166    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-44dc54\"/\"autores-model-44dc54-kserve-self-signed-certs\""
Sep 10 22:21:05.708227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.708180    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-44dc54\"/\"kube-root-ca.crt\""
Sep 10 22:21:05.708263 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.708181    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-44dc54\"/\"default-dockercfg-mxw24\""
Sep 10 22:21:05.714544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.714523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"]
Sep 10 22:21:05.839854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.839820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wk9g\" (UniqueName: \"kubernetes.io/projected/270c0e87-69d7-4377-a5ae-b1b9f4f75577-kube-api-access-9wk9g\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.839854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.839855    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tls-certs\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.840064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.839881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tmp-dir\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.840064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.839937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-model-cache\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.840064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.839989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-home\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.840064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.840009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-dshm\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.941642 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.941608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wk9g\" (UniqueName: \"kubernetes.io/projected/270c0e87-69d7-4377-a5ae-b1b9f4f75577-kube-api-access-9wk9g\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.941642 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.941646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tls-certs\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.941828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.941718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tmp-dir\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.941828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.941751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-model-cache\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.941828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.941801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-home\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.941974 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.941826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-dshm\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.942207 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.942180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tmp-dir\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.942207 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.942196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-home\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.942320 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.942238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-model-cache\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.944112 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.944091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-dshm\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.944292 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.944277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tls-certs\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:05.948812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:05.948791    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wk9g\" (UniqueName: \"kubernetes.io/projected/270c0e87-69d7-4377-a5ae-b1b9f4f75577-kube-api-access-9wk9g\") pod \"autores-model-44dc54-kserve-8645955947-gvng6\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") " pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:06.015636 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.015602    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:06.261334 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.261298    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"]
Sep 10 22:21:06.261640 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.261581    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548" podUID="308cb5ab-547d-4334-aa45-697fb6420197" containerName="payload-processing" containerID="cri-o://104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb" gracePeriod=30
Sep 10 22:21:06.269905 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.269403    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"]
Sep 10 22:21:06.269905 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.269676    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f" podUID="0a6098c1-b038-4b69-ab96-60ab5da82bf0" containerName="payload-pre-processing" containerID="cri-o://f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689" gracePeriod=30
Sep 10 22:21:06.296856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.296827    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"]
Sep 10 22:21:06.353379 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:21:06.353340    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod270c0e87_69d7_4377_a5ae_b1b9f4f75577.slice/crio-a8c81a81c9f75bc0e2aa698b287b78cfc9731c479e7be578e1a0566c83b36ed3 WatchSource:0}: Error finding container a8c81a81c9f75bc0e2aa698b287b78cfc9731c479e7be578e1a0566c83b36ed3: Status 404 returned error can't find the container with id a8c81a81c9f75bc0e2aa698b287b78cfc9731c479e7be578e1a0566c83b36ed3
Sep 10 22:21:06.512550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.512515    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:21:06.548031 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.548004    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:21:06.649139 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.649100    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm8hj\" (UniqueName: \"kubernetes.io/projected/308cb5ab-547d-4334-aa45-697fb6420197-kube-api-access-mm8hj\") pod \"308cb5ab-547d-4334-aa45-697fb6420197\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") "
Sep 10 22:21:06.649329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.649164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308cb5ab-547d-4334-aa45-697fb6420197-plugins-config-volume\") pod \"308cb5ab-547d-4334-aa45-697fb6420197\" (UID: \"308cb5ab-547d-4334-aa45-697fb6420197\") "
Sep 10 22:21:06.649329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.649233    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgd9v\" (UniqueName: \"kubernetes.io/projected/0a6098c1-b038-4b69-ab96-60ab5da82bf0-kube-api-access-jgd9v\") pod \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") "
Sep 10 22:21:06.649329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.649276    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6098c1-b038-4b69-ab96-60ab5da82bf0-plugins-config-volume\") pod \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\" (UID: \"0a6098c1-b038-4b69-ab96-60ab5da82bf0\") "
Sep 10 22:21:06.649575 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.649549    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308cb5ab-547d-4334-aa45-697fb6420197-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "308cb5ab-547d-4334-aa45-697fb6420197" (UID: "308cb5ab-547d-4334-aa45-697fb6420197"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:21:06.649737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.649703    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a6098c1-b038-4b69-ab96-60ab5da82bf0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a6098c1-b038-4b69-ab96-60ab5da82bf0" (UID: "0a6098c1-b038-4b69-ab96-60ab5da82bf0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:21:06.651515 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.651494    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6098c1-b038-4b69-ab96-60ab5da82bf0-kube-api-access-jgd9v" (OuterVolumeSpecName: "kube-api-access-jgd9v") pod "0a6098c1-b038-4b69-ab96-60ab5da82bf0" (UID: "0a6098c1-b038-4b69-ab96-60ab5da82bf0"). InnerVolumeSpecName "kube-api-access-jgd9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:06.651571 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.651542    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308cb5ab-547d-4334-aa45-697fb6420197-kube-api-access-mm8hj" (OuterVolumeSpecName: "kube-api-access-mm8hj") pod "308cb5ab-547d-4334-aa45-697fb6420197" (UID: "308cb5ab-547d-4334-aa45-697fb6420197"). InnerVolumeSpecName "kube-api-access-mm8hj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:06.750334 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.750294    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm8hj\" (UniqueName: \"kubernetes.io/projected/308cb5ab-547d-4334-aa45-697fb6420197-kube-api-access-mm8hj\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:06.750334 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.750331    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/308cb5ab-547d-4334-aa45-697fb6420197-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:06.750334 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.750341    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgd9v\" (UniqueName: \"kubernetes.io/projected/0a6098c1-b038-4b69-ab96-60ab5da82bf0-kube-api-access-jgd9v\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:06.750751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:06.750351    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a6098c1-b038-4b69-ab96-60ab5da82bf0-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:07.370767 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.370730    2563 generic.go:356] "Generic (PLEG): container finished" podID="0a6098c1-b038-4b69-ab96-60ab5da82bf0" containerID="f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689" exitCode=0
Sep 10 22:21:07.371042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.370780    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"
Sep 10 22:21:07.371042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.370816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f" event={"ID":"0a6098c1-b038-4b69-ab96-60ab5da82bf0","Type":"ContainerDied","Data":"f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689"}
Sep 10 22:21:07.371042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.370878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f" event={"ID":"0a6098c1-b038-4b69-ab96-60ab5da82bf0","Type":"ContainerDied","Data":"e02ed4131b97b06510ec4c66b8b6c5aa85925a8f67d06ec4ebdca10596b13314"}
Sep 10 22:21:07.371042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.370908    2563 scope.go:117] "RemoveContainer" containerID="f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689"
Sep 10 22:21:07.372526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.372170    2563 generic.go:356] "Generic (PLEG): container finished" podID="308cb5ab-547d-4334-aa45-697fb6420197" containerID="104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb" exitCode=0
Sep 10 22:21:07.372526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.372206    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"
Sep 10 22:21:07.372526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.372249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548" event={"ID":"308cb5ab-547d-4334-aa45-697fb6420197","Type":"ContainerDied","Data":"104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb"}
Sep 10 22:21:07.372526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.372294    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548" event={"ID":"308cb5ab-547d-4334-aa45-697fb6420197","Type":"ContainerDied","Data":"89ac268a9b6e4c033e8a63d3ae834ddedba2c154fca38437f1b7d84d0b4dce20"}
Sep 10 22:21:07.374320 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.374281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6" event={"ID":"270c0e87-69d7-4377-a5ae-b1b9f4f75577","Type":"ContainerStarted","Data":"7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9"}
Sep 10 22:21:07.374486 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.374328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6" event={"ID":"270c0e87-69d7-4377-a5ae-b1b9f4f75577","Type":"ContainerStarted","Data":"a8c81a81c9f75bc0e2aa698b287b78cfc9731c479e7be578e1a0566c83b36ed3"}
Sep 10 22:21:07.380283 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.380266    2563 scope.go:117] "RemoveContainer" containerID="f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689"
Sep 10 22:21:07.380576 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:07.380555    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689\": container with ID starting with f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689 not found: ID does not exist" containerID="f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689"
Sep 10 22:21:07.380650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.380586    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689"} err="failed to get container status \"f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689\": rpc error: code = NotFound desc = could not find container \"f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689\": container with ID starting with f2be9f50f550f29acfafa64cdd244c264182a23379168be474c5a5c38097b689 not found: ID does not exist"
Sep 10 22:21:07.380650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.380606    2563 scope.go:117] "RemoveContainer" containerID="104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb"
Sep 10 22:21:07.388492 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.388428    2563 scope.go:117] "RemoveContainer" containerID="104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb"
Sep 10 22:21:07.388775 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:07.388757    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb\": container with ID starting with 104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb not found: ID does not exist" containerID="104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb"
Sep 10 22:21:07.388841 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.388782    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb"} err="failed to get container status \"104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb\": rpc error: code = NotFound desc = could not find container \"104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb\": container with ID starting with 104197a4870cdf7519b7362e51d7ccc0eea09a77847ed00765cc9eb5959884bb not found: ID does not exist"
Sep 10 22:21:07.391228 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.391190    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6" podStartSLOduration=2.3911753239999998 podStartE2EDuration="2.391175324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:21:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:21:07.39067276 +0000 UTC m=+936.775178182" watchObservedRunningTime="2026-09-10 22:21:07.391175324 +0000 UTC m=+936.775680751"
Sep 10 22:21:07.403748 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.403727    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"]
Sep 10 22:21:07.407228 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.407206    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-178c50fc-78d74ff8b-hrp4f"]
Sep 10 22:21:07.417135 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.417111    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"]
Sep 10 22:21:07.420690 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:07.420669    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-178c50fc-7f554495b9-sr548"]
Sep 10 22:21:08.364335 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.364298    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"]
Sep 10 22:21:08.364925 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.364903    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a6098c1-b038-4b69-ab96-60ab5da82bf0" containerName="payload-pre-processing"
Sep 10 22:21:08.364998 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.364929    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a6098c1-b038-4b69-ab96-60ab5da82bf0" containerName="payload-pre-processing"
Sep 10 22:21:08.364998 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.364942    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308cb5ab-547d-4334-aa45-697fb6420197" containerName="payload-processing"
Sep 10 22:21:08.364998 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.364951    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="308cb5ab-547d-4334-aa45-697fb6420197" containerName="payload-processing"
Sep 10 22:21:08.365155 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.365061    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a6098c1-b038-4b69-ab96-60ab5da82bf0" containerName="payload-pre-processing"
Sep 10 22:21:08.365155 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.365078    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="308cb5ab-547d-4334-aa45-697fb6420197" containerName="payload-processing"
Sep 10 22:21:08.368088 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.368053    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.370341 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.370323    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2057d50e-openshift-default-dockercfg-6nrgr\""
Sep 10 22:21:08.381888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.381816    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"]
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-envoy\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-data\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467360    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-certs\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istiod-ca-cert\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-credential-socket\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwq7q\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-kube-api-access-vwq7q\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-podinfo\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-socket\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.467771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.467649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-token\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568468 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-data\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-certs\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istiod-ca-cert\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-credential-socket\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwq7q\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-kube-api-access-vwq7q\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-podinfo\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568957 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-socket\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568957 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-data\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.568957 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.568875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-certs\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.569118 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.569022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-credential-socket\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.569118 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.569087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-token\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.569228 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.569150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-socket\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.569228 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.569167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-envoy\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.570158 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.570119    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istiod-ca-cert\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.571494 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.571476    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-podinfo\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.571569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.571523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-envoy\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.579423 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.579391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-token\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.581825 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.581804    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwq7q\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-kube-api-access-vwq7q\") pod \"e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") " pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.683352 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.682980    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:08.956789 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.956768    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"]
Sep 10 22:21:08.959031 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:21:08.959009    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a2ed67_3cc7_410e_85ce_0a4648ec7972.slice/crio-9ffa7bbdf18e64d1a5edd14dda102eac0f582ba467c076f7ba80b3972c4ddeb2 WatchSource:0}: Error finding container 9ffa7bbdf18e64d1a5edd14dda102eac0f582ba467c076f7ba80b3972c4ddeb2: Status 404 returned error can't find the container with id 9ffa7bbdf18e64d1a5edd14dda102eac0f582ba467c076f7ba80b3972c4ddeb2
Sep 10 22:21:08.961253 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.961224    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:21:08.961309 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.961287    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:21:08.961349 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:08.961317    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:21:09.111406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:09.111374    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a6098c1-b038-4b69-ab96-60ab5da82bf0" path="/var/lib/kubelet/pods/0a6098c1-b038-4b69-ab96-60ab5da82bf0/volumes"
Sep 10 22:21:09.111841 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:09.111825    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308cb5ab-547d-4334-aa45-697fb6420197" path="/var/lib/kubelet/pods/308cb5ab-547d-4334-aa45-697fb6420197/volumes"
Sep 10 22:21:09.390831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:09.390799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" event={"ID":"a4a2ed67-3cc7-410e-85ce-0a4648ec7972","Type":"ContainerStarted","Data":"f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7"}
Sep 10 22:21:09.391219 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:09.390852    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" event={"ID":"a4a2ed67-3cc7-410e-85ce-0a4648ec7972","Type":"ContainerStarted","Data":"9ffa7bbdf18e64d1a5edd14dda102eac0f582ba467c076f7ba80b3972c4ddeb2"}
Sep 10 22:21:09.408562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:09.408512    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" podStartSLOduration=1.408495133 podStartE2EDuration="1.408495133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:21:09.406938092 +0000 UTC m=+938.791443516" watchObservedRunningTime="2026-09-10 22:21:09.408495133 +0000 UTC m=+938.793000554"
Sep 10 22:21:09.683527 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:09.683427    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:10.689107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:10.689077    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:11.401833 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:11.401806    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:11.402894 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:11.402867    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:16.016093 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:16.016049    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:16.016498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:16.016347    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:16.028747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:16.028722    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:16.441876 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:16.441800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:17.186044 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.186010    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"]
Sep 10 22:21:17.188851 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.188831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.190978 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.190958    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-669e2b\"/\"openshift-service-ca.crt\""
Sep 10 22:21:17.191085 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.191026    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-669e2b\"/\"kube-root-ca.crt\""
Sep 10 22:21:17.191623 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.191597    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-669e2b\"/\"default-dockercfg-c2lx4\""
Sep 10 22:21:17.191871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.191851    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-669e2b\"/\"test-model-669e2b-kserve-self-signed-certs\""
Sep 10 22:21:17.197823 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.197802    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"]
Sep 10 22:21:17.254521 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.254493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c118dc44-4f67-4563-afb0-d14e6fe1487f-tls-certs\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.254664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.254534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-home\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.254664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.254587    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-model-cache\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.254664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.254620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm6bv\" (UniqueName: \"kubernetes.io/projected/c118dc44-4f67-4563-afb0-d14e6fe1487f-kube-api-access-sm6bv\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.254778 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.254695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-tmp-dir\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.254778 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.254731    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-dshm\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.355842 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.355796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-model-cache\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.355851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm6bv\" (UniqueName: \"kubernetes.io/projected/c118dc44-4f67-4563-afb0-d14e6fe1487f-kube-api-access-sm6bv\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.355912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-tmp-dir\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.355951    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-dshm\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356169 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.356050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c118dc44-4f67-4563-afb0-d14e6fe1487f-tls-certs\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356169 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.356087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-home\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356284 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.356199    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-model-cache\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356284 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.356224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-tmp-dir\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.356400 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.356377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-home\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.358303 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.358283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-dshm\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.358596 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.358577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c118dc44-4f67-4563-afb0-d14e6fe1487f-tls-certs\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.363077 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.363054    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm6bv\" (UniqueName: \"kubernetes.io/projected/c118dc44-4f67-4563-afb0-d14e6fe1487f-kube-api-access-sm6bv\") pod \"test-model-669e2b-kserve-6fb6cfb589-2btcf\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") " pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.500711 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.500679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:17.769144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:17.769120    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"]
Sep 10 22:21:17.771524 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:21:17.771500    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc118dc44_4f67_4563_afb0_d14e6fe1487f.slice/crio-defcbfa75ab09ecb1cbb38faf67918c69ef0386db2465ed50e9fa772b6634f52 WatchSource:0}: Error finding container defcbfa75ab09ecb1cbb38faf67918c69ef0386db2465ed50e9fa772b6634f52: Status 404 returned error can't find the container with id defcbfa75ab09ecb1cbb38faf67918c69ef0386db2465ed50e9fa772b6634f52
Sep 10 22:21:18.442613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:18.442580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf" event={"ID":"c118dc44-4f67-4563-afb0-d14e6fe1487f","Type":"ContainerStarted","Data":"4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6"}
Sep 10 22:21:18.442983 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:18.442627    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf" event={"ID":"c118dc44-4f67-4563-afb0-d14e6fe1487f","Type":"ContainerStarted","Data":"defcbfa75ab09ecb1cbb38faf67918c69ef0386db2465ed50e9fa772b6634f52"}
Sep 10 22:21:18.459860 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:18.459810    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf" podStartSLOduration=1.459794893 podStartE2EDuration="1.459794893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:21:18.457280095 +0000 UTC m=+947.841785520" watchObservedRunningTime="2026-09-10 22:21:18.459794893 +0000 UTC m=+947.844300362"
Sep 10 22:21:23.126603 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:23.126562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:21:25.261970 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:25.261939    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"]
Sep 10 22:21:27.501031 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:27.500995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:27.501557 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:27.501055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:27.538162 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:27.538117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:28.011103 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:28.011069    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"]
Sep 10 22:21:28.505088 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:28.505061    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:21:32.424084 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:32.424048    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"]
Sep 10 22:21:33.522094 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:33.522063    2563 generic.go:356] "Generic (PLEG): container finished" podID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerID="f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7" exitCode=0
Sep 10 22:21:33.522555 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:33.522141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" event={"ID":"a4a2ed67-3cc7-410e-85ce-0a4648ec7972","Type":"ContainerDied","Data":"f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7"}
Sep 10 22:21:33.522627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:33.522613    2563 scope.go:117] "RemoveContainer" containerID="f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7"
Sep 10 22:21:33.523663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:33.523638    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:21:33.523728 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:33.523706    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:21:33.523763 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:33.523737    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:21:34.533391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:34.533345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" event={"ID":"a4a2ed67-3cc7-410e-85ce-0a4648ec7972","Type":"ContainerStarted","Data":"c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59"}
Sep 10 22:21:34.684489 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:34.684423    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:34.684663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:34.684641    2563 patch_prober.go:28] interesting pod/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 22:21:34.684710 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:34.684693    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 22:21:35.684095 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:35.684057    2563 patch_prober.go:28] interesting pod/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 22:21:35.684507 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:35.684119    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 22:21:36.684313 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:36.684280    2563 patch_prober.go:28] interesting pod/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 22:21:36.684715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:36.684340    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 22:21:37.683976 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:37.683938    2563 patch_prober.go:28] interesting pod/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 22:21:37.684147 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:37.683997    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 22:21:38.683889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:38.683856    2563 patch_prober.go:28] interesting pod/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 22:21:38.684335 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:38.683910    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 22:21:38.978298 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:38.978219    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"]
Sep 10 22:21:38.978675 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:38.978504    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b" podUID="d342270c-0502-46ca-8d91-91785e9feecc" containerName="payload-processing" containerID="cri-o://c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d" gracePeriod=30
Sep 10 22:21:39.303995 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.303971    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:21:39.383068 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.383033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqhn9\" (UniqueName: \"kubernetes.io/projected/d342270c-0502-46ca-8d91-91785e9feecc-kube-api-access-hqhn9\") pod \"d342270c-0502-46ca-8d91-91785e9feecc\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") "
Sep 10 22:21:39.383261 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.383130    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d342270c-0502-46ca-8d91-91785e9feecc-plugins-config-volume\") pod \"d342270c-0502-46ca-8d91-91785e9feecc\" (UID: \"d342270c-0502-46ca-8d91-91785e9feecc\") "
Sep 10 22:21:39.383524 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.383499    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d342270c-0502-46ca-8d91-91785e9feecc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d342270c-0502-46ca-8d91-91785e9feecc" (UID: "d342270c-0502-46ca-8d91-91785e9feecc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:21:39.385282 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.385252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d342270c-0502-46ca-8d91-91785e9feecc-kube-api-access-hqhn9" (OuterVolumeSpecName: "kube-api-access-hqhn9") pod "d342270c-0502-46ca-8d91-91785e9feecc" (UID: "d342270c-0502-46ca-8d91-91785e9feecc"). InnerVolumeSpecName "kube-api-access-hqhn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:39.483795 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.483766    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqhn9\" (UniqueName: \"kubernetes.io/projected/d342270c-0502-46ca-8d91-91785e9feecc-kube-api-access-hqhn9\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:39.483795 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.483793    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d342270c-0502-46ca-8d91-91785e9feecc-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:39.563043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.562949    2563 generic.go:356] "Generic (PLEG): container finished" podID="d342270c-0502-46ca-8d91-91785e9feecc" containerID="c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d" exitCode=0
Sep 10 22:21:39.563043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.562995    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"
Sep 10 22:21:39.563294 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.563035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b" event={"ID":"d342270c-0502-46ca-8d91-91785e9feecc","Type":"ContainerDied","Data":"c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d"}
Sep 10 22:21:39.563294 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.563094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b" event={"ID":"d342270c-0502-46ca-8d91-91785e9feecc","Type":"ContainerDied","Data":"21c772b3ee61c41262d5f81a9172c3cc72f8e4365e7797ed4567fe3dd696049d"}
Sep 10 22:21:39.563294 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.563125    2563 scope.go:117] "RemoveContainer" containerID="c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d"
Sep 10 22:21:39.576538 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.576508    2563 scope.go:117] "RemoveContainer" containerID="c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d"
Sep 10 22:21:39.577208 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:39.577175    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d\": container with ID starting with c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d not found: ID does not exist" containerID="c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d"
Sep 10 22:21:39.577315 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.577217    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d"} err="failed to get container status \"c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d\": rpc error: code = NotFound desc = could not find container \"c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d\": container with ID starting with c886519692e79e6900b617a31e0a8e142f25b0b42dc5bd5f2d32972c529c230d not found: ID does not exist"
Sep 10 22:21:39.589609 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.589580    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"]
Sep 10 22:21:39.594823 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.594780    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-44dc54-65c7f88d49-f2l7b"]
Sep 10 22:21:39.716033 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.716006    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:39.716550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.716431    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:39.718337 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:39.718316    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:40.281628 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.281584    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2" podUID="c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" containerName="main" containerID="cri-o://ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29" gracePeriod=45
Sep 10 22:21:40.531106 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.531083    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:21:40.576849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.576752    2563 generic.go:356] "Generic (PLEG): container finished" podID="c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" containerID="ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29" exitCode=0
Sep 10 22:21:40.576849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.576827    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"
Sep 10 22:21:40.576849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.576838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2" event={"ID":"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb","Type":"ContainerDied","Data":"ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29"}
Sep 10 22:21:40.577089 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.576884    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2" event={"ID":"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb","Type":"ContainerDied","Data":"bf32bb6a56bef00144d760a263555dc90793114a387d432aef8fd349b403c4d2"}
Sep 10 22:21:40.577089 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.576907    2563 scope.go:117] "RemoveContainer" containerID="ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29"
Sep 10 22:21:40.585303 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.585286    2563 scope.go:117] "RemoveContainer" containerID="ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29"
Sep 10 22:21:40.585594 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:40.585574    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29\": container with ID starting with ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29 not found: ID does not exist" containerID="ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29"
Sep 10 22:21:40.585649 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.585602    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29"} err="failed to get container status \"ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29\": rpc error: code = NotFound desc = could not find container \"ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29\": container with ID starting with ba9b030764c61edf03b23b8b7fa7424a365061e428161d33cf9bd853b673ea29 not found: ID does not exist"
Sep 10 22:21:40.694973 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.694941    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-dshm\") pod \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") "
Sep 10 22:21:40.695144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.694988    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-model-cache\") pod \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") "
Sep 10 22:21:40.695144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695010    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tmp-dir\") pod \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") "
Sep 10 22:21:40.695144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695029    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tls-certs\") pod \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") "
Sep 10 22:21:40.695144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695068    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqk9s\" (UniqueName: \"kubernetes.io/projected/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-kube-api-access-tqk9s\") pod \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") "
Sep 10 22:21:40.695365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-home\") pod \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\" (UID: \"c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb\") "
Sep 10 22:21:40.695365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695255    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" (UID: "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:40.695365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695261    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" (UID: "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:40.695564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695504    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-home" (OuterVolumeSpecName: "home") pod "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" (UID: "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:40.695720 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695658    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:40.695720 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695684    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:40.695720 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.695698    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:40.697489 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.697455    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-dshm" (OuterVolumeSpecName: "dshm") pod "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" (UID: "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:40.697489 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.697478    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-kube-api-access-tqk9s" (OuterVolumeSpecName: "kube-api-access-tqk9s") pod "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" (UID: "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb"). InnerVolumeSpecName "kube-api-access-tqk9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:40.697611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.697515    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" (UID: "c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:21:40.796356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.796324    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:40.796356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.796351    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqk9s\" (UniqueName: \"kubernetes.io/projected/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-kube-api-access-tqk9s\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:40.796356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.796361    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:40.899985 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.899949    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"]
Sep 10 22:21:40.904410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:40.904385    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-7660dbe4-kserve-78b9d59d4c-lwln2"]
Sep 10 22:21:41.112187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:41.112151    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" path="/var/lib/kubelet/pods/c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb/volumes"
Sep 10 22:21:41.112626 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:41.112606    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d342270c-0502-46ca-8d91-91785e9feecc" path="/var/lib/kubelet/pods/d342270c-0502-46ca-8d91-91785e9feecc/volumes"
Sep 10 22:21:41.405346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:41.405312    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:21:43.051654 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.051605    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6" podUID="270c0e87-69d7-4377-a5ae-b1b9f4f75577" containerName="main" containerID="cri-o://7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9" gracePeriod=45
Sep 10 22:21:43.153336 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.153300    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"]
Sep 10 22:21:43.153546 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.153530    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy" containerID="cri-o://c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59" gracePeriod=30
Sep 10 22:21:43.311895 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.311844    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:43.421960 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.421928    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tls-certs\") pod \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") "
Sep 10 22:21:43.422132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.421989    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wk9g\" (UniqueName: \"kubernetes.io/projected/270c0e87-69d7-4377-a5ae-b1b9f4f75577-kube-api-access-9wk9g\") pod \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") "
Sep 10 22:21:43.422132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422014    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-model-cache\") pod \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") "
Sep 10 22:21:43.422132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-dshm\") pod \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") "
Sep 10 22:21:43.422307 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tmp-dir\") pod \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") "
Sep 10 22:21:43.422307 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422186    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-home\") pod \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\" (UID: \"270c0e87-69d7-4377-a5ae-b1b9f4f75577\") "
Sep 10 22:21:43.422415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422328    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-model-cache" (OuterVolumeSpecName: "model-cache") pod "270c0e87-69d7-4377-a5ae-b1b9f4f75577" (UID: "270c0e87-69d7-4377-a5ae-b1b9f4f75577"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:43.422515 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422417    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "270c0e87-69d7-4377-a5ae-b1b9f4f75577" (UID: "270c0e87-69d7-4377-a5ae-b1b9f4f75577"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:43.422585 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422568    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-home" (OuterVolumeSpecName: "home") pod "270c0e87-69d7-4377-a5ae-b1b9f4f75577" (UID: "270c0e87-69d7-4377-a5ae-b1b9f4f75577"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:43.422633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422604    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:43.422633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.422625    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:43.424429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.424408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-dshm" (OuterVolumeSpecName: "dshm") pod "270c0e87-69d7-4377-a5ae-b1b9f4f75577" (UID: "270c0e87-69d7-4377-a5ae-b1b9f4f75577"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:43.424543 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.424422    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "270c0e87-69d7-4377-a5ae-b1b9f4f75577" (UID: "270c0e87-69d7-4377-a5ae-b1b9f4f75577"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:21:43.424543 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.424430    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/270c0e87-69d7-4377-a5ae-b1b9f4f75577-kube-api-access-9wk9g" (OuterVolumeSpecName: "kube-api-access-9wk9g") pod "270c0e87-69d7-4377-a5ae-b1b9f4f75577" (UID: "270c0e87-69d7-4377-a5ae-b1b9f4f75577"). InnerVolumeSpecName "kube-api-access-9wk9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:43.523659 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.523627    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:43.523659 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.523656    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/270c0e87-69d7-4377-a5ae-b1b9f4f75577-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:43.523659 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.523665    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wk9g\" (UniqueName: \"kubernetes.io/projected/270c0e87-69d7-4377-a5ae-b1b9f4f75577-kube-api-access-9wk9g\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:43.523883 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.523673    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/270c0e87-69d7-4377-a5ae-b1b9f4f75577-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:43.596081 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.595999    2563 generic.go:356] "Generic (PLEG): container finished" podID="270c0e87-69d7-4377-a5ae-b1b9f4f75577" containerID="7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9" exitCode=0
Sep 10 22:21:43.596081 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.596043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6" event={"ID":"270c0e87-69d7-4377-a5ae-b1b9f4f75577","Type":"ContainerDied","Data":"7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9"}
Sep 10 22:21:43.596081 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.596050    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"
Sep 10 22:21:43.596081 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.596080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6" event={"ID":"270c0e87-69d7-4377-a5ae-b1b9f4f75577","Type":"ContainerDied","Data":"a8c81a81c9f75bc0e2aa698b287b78cfc9731c479e7be578e1a0566c83b36ed3"}
Sep 10 22:21:43.596390 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.596096    2563 scope.go:117] "RemoveContainer" containerID="7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9"
Sep 10 22:21:43.605585 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.605564    2563 scope.go:117] "RemoveContainer" containerID="7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9"
Sep 10 22:21:43.605875 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:43.605856    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9\": container with ID starting with 7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9 not found: ID does not exist" containerID="7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9"
Sep 10 22:21:43.605942 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.605885    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9"} err="failed to get container status \"7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9\": rpc error: code = NotFound desc = could not find container \"7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9\": container with ID starting with 7a9aa5fdb99869e5c16d7d4b32b711770dd451c42b45c0bb89010a76ad1cccf9 not found: ID does not exist"
Sep 10 22:21:43.617861 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.617834    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"]
Sep 10 22:21:43.621668 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:43.621646    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-44dc54/autores-model-44dc54-kserve-8645955947-gvng6"]
Sep 10 22:21:45.111017 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:45.110975    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="270c0e87-69d7-4377-a5ae-b1b9f4f75577" path="/var/lib/kubelet/pods/270c0e87-69d7-4377-a5ae-b1b9f4f75577/volumes"
Sep 10 22:21:46.029782 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:46.029750    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"]
Sep 10 22:21:46.030060 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:46.029996    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy" containerID="cri-o://9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca" gracePeriod=30
Sep 10 22:21:47.444164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.444125    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6" podUID="df6ff56a-8b6d-4553-84ed-d783d2e837b8" containerName="main" containerID="cri-o://2f9ce69d9a836609205e068efdfab9e8e43fe7a0756b24fe6f03cc5945fc4a11" gracePeriod=45
Sep 10 22:21:47.626299 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.626261    2563 generic.go:356] "Generic (PLEG): container finished" podID="df6ff56a-8b6d-4553-84ed-d783d2e837b8" containerID="2f9ce69d9a836609205e068efdfab9e8e43fe7a0756b24fe6f03cc5945fc4a11" exitCode=0
Sep 10 22:21:47.626494 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.626329    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6" event={"ID":"df6ff56a-8b6d-4553-84ed-d783d2e837b8","Type":"ContainerDied","Data":"2f9ce69d9a836609205e068efdfab9e8e43fe7a0756b24fe6f03cc5945fc4a11"}
Sep 10 22:21:47.693473 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.693429    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:21:47.865670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.865642    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-dshm\") pod \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") "
Sep 10 22:21:47.865858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.865715    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-model-cache\") pod \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") "
Sep 10 22:21:47.865858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.865740    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tls-certs\") pod \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") "
Sep 10 22:21:47.865858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.865761    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tmp-dir\") pod \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") "
Sep 10 22:21:47.865858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.865785    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-home\") pod \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") "
Sep 10 22:21:47.865858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.865822    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lsqw\" (UniqueName: \"kubernetes.io/projected/df6ff56a-8b6d-4553-84ed-d783d2e837b8-kube-api-access-9lsqw\") pod \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\" (UID: \"df6ff56a-8b6d-4553-84ed-d783d2e837b8\") "
Sep 10 22:21:47.866128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.865993    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df6ff56a-8b6d-4553-84ed-d783d2e837b8" (UID: "df6ff56a-8b6d-4553-84ed-d783d2e837b8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:47.866128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.866030    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-model-cache" (OuterVolumeSpecName: "model-cache") pod "df6ff56a-8b6d-4553-84ed-d783d2e837b8" (UID: "df6ff56a-8b6d-4553-84ed-d783d2e837b8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:47.866128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.866057    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-home" (OuterVolumeSpecName: "home") pod "df6ff56a-8b6d-4553-84ed-d783d2e837b8" (UID: "df6ff56a-8b6d-4553-84ed-d783d2e837b8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:47.866302 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.866220    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:47.866302 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.866231    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:47.866302 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.866244    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:47.868040 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.868018    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-dshm" (OuterVolumeSpecName: "dshm") pod "df6ff56a-8b6d-4553-84ed-d783d2e837b8" (UID: "df6ff56a-8b6d-4553-84ed-d783d2e837b8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:47.868040 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.868016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df6ff56a-8b6d-4553-84ed-d783d2e837b8" (UID: "df6ff56a-8b6d-4553-84ed-d783d2e837b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:21:47.868040 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.868030    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df6ff56a-8b6d-4553-84ed-d783d2e837b8-kube-api-access-9lsqw" (OuterVolumeSpecName: "kube-api-access-9lsqw") pod "df6ff56a-8b6d-4553-84ed-d783d2e837b8" (UID: "df6ff56a-8b6d-4553-84ed-d783d2e837b8"). InnerVolumeSpecName "kube-api-access-9lsqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:47.967415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.967382    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df6ff56a-8b6d-4553-84ed-d783d2e837b8-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:47.967415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.967407    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df6ff56a-8b6d-4553-84ed-d783d2e837b8-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:47.967415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:47.967418    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lsqw\" (UniqueName: \"kubernetes.io/projected/df6ff56a-8b6d-4553-84ed-d783d2e837b8-kube-api-access-9lsqw\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.400166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.400142    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:48.573009 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.572973    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwq7q\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-kube-api-access-vwq7q\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573022    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-socket\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573044    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-data\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-token\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istiod-ca-cert\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573133    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-envoy\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-certs\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573213    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-credential-socket\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573278    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-podinfo\") pod \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\" (UID: \"a4a2ed67-3cc7-410e-85ce-0a4648ec7972\") "
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573279    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:48.573472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573466    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:48.573963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573474    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:48.573963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573545    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:48.573963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573659    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.573963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573678    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.573963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573692    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.573963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573705    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.573963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.573658    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:21:48.575475 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.575431    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-kube-api-access-vwq7q" (OuterVolumeSpecName: "kube-api-access-vwq7q") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "kube-api-access-vwq7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:48.575715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.575697    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:21:48.576050 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.576025    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:48.576164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.576059    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4a2ed67-3cc7-410e-85ce-0a4648ec7972" (UID: "a4a2ed67-3cc7-410e-85ce-0a4648ec7972"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:48.632525 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.632492    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"
Sep 10 22:21:48.632696 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.632496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6" event={"ID":"df6ff56a-8b6d-4553-84ed-d783d2e837b8","Type":"ContainerDied","Data":"a853593eab2644bd85e9a2bb8cbb9f260f83bc776a43158f57496f50c512dbd6"}
Sep 10 22:21:48.632696 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.632623    2563 scope.go:117] "RemoveContainer" containerID="2f9ce69d9a836609205e068efdfab9e8e43fe7a0756b24fe6f03cc5945fc4a11"
Sep 10 22:21:48.634544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.634518    2563 generic.go:356] "Generic (PLEG): container finished" podID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerID="c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59" exitCode=0
Sep 10 22:21:48.634655 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.634545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" event={"ID":"a4a2ed67-3cc7-410e-85ce-0a4648ec7972","Type":"ContainerDied","Data":"c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59"}
Sep 10 22:21:48.634655 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.634603    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w" event={"ID":"a4a2ed67-3cc7-410e-85ce-0a4648ec7972","Type":"ContainerDied","Data":"9ffa7bbdf18e64d1a5edd14dda102eac0f582ba467c076f7ba80b3972c4ddeb2"}
Sep 10 22:21:48.634655 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.634590    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"
Sep 10 22:21:48.643887 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.643868    2563 scope.go:117] "RemoveContainer" containerID="c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59"
Sep 10 22:21:48.652638 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.652618    2563 scope.go:117] "RemoveContainer" containerID="f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7"
Sep 10 22:21:48.653328 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.653310    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"]
Sep 10 22:21:48.657381 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.657359    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-7660dbe4-kserve-6857979fc7-8whc6"]
Sep 10 22:21:48.661132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.661104    2563 scope.go:117] "RemoveContainer" containerID="c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59"
Sep 10 22:21:48.661410 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:48.661394    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59\": container with ID starting with c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59 not found: ID does not exist" containerID="c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59"
Sep 10 22:21:48.661474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.661422    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59"} err="failed to get container status \"c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59\": rpc error: code = NotFound desc = could not find container \"c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59\": container with ID starting with c4f403df36d00fcc510c719a106f8677ab29141fe42d666b0c7f89a8f4c9db59 not found: ID does not exist"
Sep 10 22:21:48.661474 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.661457    2563 scope.go:117] "RemoveContainer" containerID="f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7"
Sep 10 22:21:48.661751 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:48.661731    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7\": container with ID starting with f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7 not found: ID does not exist" containerID="f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7"
Sep 10 22:21:48.661808 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.661757    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7"} err="failed to get container status \"f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7\": rpc error: code = NotFound desc = could not find container \"f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7\": container with ID starting with f50581dbfce3a4cfaaf2a1aeb37cba22754045fa480c971f3f4265f68fefa2d7 not found: ID does not exist"
Sep 10 22:21:48.666791 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.666767    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"]
Sep 10 22:21:48.669135 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.669116    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2057d50e-openshift-default-5b56688444-hzh4w"]
Sep 10 22:21:48.674699 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.674679    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.674699 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.674700    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwq7q\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-kube-api-access-vwq7q\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.674810 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.674709    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.674810 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.674718    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:48.674810 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:48.674725    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4a2ed67-3cc7-410e-85ce-0a4648ec7972-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:49.110871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:49.110838    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" path="/var/lib/kubelet/pods/a4a2ed67-3cc7-410e-85ce-0a4648ec7972/volumes"
Sep 10 22:21:49.111235 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:49.111222    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df6ff56a-8b6d-4553-84ed-d783d2e837b8" path="/var/lib/kubelet/pods/df6ff56a-8b6d-4553-84ed-d783d2e837b8/volumes"
Sep 10 22:21:51.290282 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.290221    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:21:51.294351 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294330    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-socket\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294365    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-token\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294393    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-credential-socket\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294515 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294491    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-certs\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294542    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx6p4\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-kube-api-access-rx6p4\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294582    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-data\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294612    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-envoy\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294617    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:51.294679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294644    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istiod-ca-cert\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.294837 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294747    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:51.294837 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294800    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:51.294946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294845    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-data" (OuterVolumeSpecName: "istio-data") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:51.294946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294873    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-podinfo\") pod \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\" (UID: \"129e09c3-9ae9-4c39-9493-ea5a2654b5aa\") "
Sep 10 22:21:51.295243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.294955    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:21:51.295243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.295200    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.295243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.295223    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.295243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.295237    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.295522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.295249    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.295522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.295261    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.297492 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.297429    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-token" (OuterVolumeSpecName: "istio-token") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:51.297492 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.297468    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-kube-api-access-rx6p4" (OuterVolumeSpecName: "kube-api-access-rx6p4") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "kube-api-access-rx6p4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:21:51.297670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.297586    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:21:51.297670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.297589    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "129e09c3-9ae9-4c39-9493-ea5a2654b5aa" (UID: "129e09c3-9ae9-4c39-9493-ea5a2654b5aa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:21:51.298008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.297989    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"]
Sep 10 22:21:51.298332 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298318    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298332    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298343    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d342270c-0502-46ca-8d91-91785e9feecc" containerName="payload-processing"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298348    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d342270c-0502-46ca-8d91-91785e9feecc" containerName="payload-processing"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298356    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" containerName="main"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298361    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" containerName="main"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298370    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298376    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298387    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="270c0e87-69d7-4377-a5ae-b1b9f4f75577" containerName="main"
Sep 10 22:21:51.298395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298392    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="270c0e87-69d7-4377-a5ae-b1b9f4f75577" containerName="main"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298404    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298409    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298415    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df6ff56a-8b6d-4553-84ed-d783d2e837b8" containerName="main"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298420    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df6ff56a-8b6d-4553-84ed-d783d2e837b8" containerName="main"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298425    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298430    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298504    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298513    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d342270c-0502-46ca-8d91-91785e9feecc" containerName="payload-processing"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298520    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df6ff56a-8b6d-4553-84ed-d783d2e837b8" containerName="main"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298525    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="270c0e87-69d7-4377-a5ae-b1b9f4f75577" containerName="main"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298529    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298535    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c672ac99-0c4a-4c5c-bd42-d1192bc0f4cb" containerName="main"
Sep 10 22:21:51.298866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.298541    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerName="istio-proxy"
Sep 10 22:21:51.301796 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.301758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.304812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.304793    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-65b791\""
Sep 10 22:21:51.318047 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.318023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"]
Sep 10 22:21:51.338737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.338707    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"]
Sep 10 22:21:51.339114 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.339102    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4a2ed67-3cc7-410e-85ce-0a4648ec7972" containerName="istio-proxy"
Sep 10 22:21:51.342047 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.342005    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.344312 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.344287    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-65b791-dockercfg-sgmsr\""
Sep 10 22:21:51.353525 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.353498    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"]
Sep 10 22:21:51.396024 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.395991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c88942-ba0d-4e25-a02e-be37d11fc14e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") " pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.396187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.396050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq5gh\" (UniqueName: \"kubernetes.io/projected/88c88942-ba0d-4e25-a02e-be37d11fc14e-kube-api-access-hq5gh\") pod \"payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") " pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.396187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.396130    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d991a2-b436-48b8-99de-9e47900514ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.396262 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.396210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh2jt\" (UniqueName: \"kubernetes.io/projected/a8d991a2-b436-48b8-99de-9e47900514ba-kube-api-access-mh2jt\") pod \"payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.396262 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.396253    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx6p4\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-kube-api-access-rx6p4\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.396333 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.396269    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.396333 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.396278    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.396333 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.396286    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/129e09c3-9ae9-4c39-9493-ea5a2654b5aa-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:21:51.497560 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.497516    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq5gh\" (UniqueName: \"kubernetes.io/projected/88c88942-ba0d-4e25-a02e-be37d11fc14e-kube-api-access-hq5gh\") pod \"payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") " pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.497767 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.497599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d991a2-b436-48b8-99de-9e47900514ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.497767 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.497693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh2jt\" (UniqueName: \"kubernetes.io/projected/a8d991a2-b436-48b8-99de-9e47900514ba-kube-api-access-mh2jt\") pod \"payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.497767 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.497725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c88942-ba0d-4e25-a02e-be37d11fc14e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") " pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.498300 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.498278    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d991a2-b436-48b8-99de-9e47900514ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.498650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.498629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c88942-ba0d-4e25-a02e-be37d11fc14e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") " pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.505799 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.505775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq5gh\" (UniqueName: \"kubernetes.io/projected/88c88942-ba0d-4e25-a02e-be37d11fc14e-kube-api-access-hq5gh\") pod \"payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") " pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.505914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.505798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh2jt\" (UniqueName: \"kubernetes.io/projected/a8d991a2-b436-48b8-99de-9e47900514ba-kube-api-access-mh2jt\") pod \"payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.621368 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.621286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:51.652636 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.652574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:51.654702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.654667    2563 generic.go:356] "Generic (PLEG): container finished" podID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" containerID="9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca" exitCode=0
Sep 10 22:21:51.654842 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.654708    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"
Sep 10 22:21:51.654842 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.654757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" event={"ID":"129e09c3-9ae9-4c39-9493-ea5a2654b5aa","Type":"ContainerDied","Data":"9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca"}
Sep 10 22:21:51.654842 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.654820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd" event={"ID":"129e09c3-9ae9-4c39-9493-ea5a2654b5aa","Type":"ContainerDied","Data":"c9eda03f4ae372d72c2dc48e1a8dba857e98dda62504db76c7df4612c5b803dd"}
Sep 10 22:21:51.655021 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.654874    2563 scope.go:117] "RemoveContainer" containerID="9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca"
Sep 10 22:21:51.669702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.669678    2563 scope.go:117] "RemoveContainer" containerID="ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31"
Sep 10 22:21:51.682954 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.681840    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"]
Sep 10 22:21:51.685706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.683792    2563 scope.go:117] "RemoveContainer" containerID="9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca"
Sep 10 22:21:51.685706 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:51.684271    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca\": container with ID starting with 9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca not found: ID does not exist" containerID="9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca"
Sep 10 22:21:51.685706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.684304    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca"} err="failed to get container status \"9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca\": rpc error: code = NotFound desc = could not find container \"9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca\": container with ID starting with 9b1cdd96640eed4fec98bcea0212f340e3a12b34f673b6a8ab51920619dfbaca not found: ID does not exist"
Sep 10 22:21:51.685706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.684328    2563 scope.go:117] "RemoveContainer" containerID="ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31"
Sep 10 22:21:51.685706 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:21:51.684769    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31\": container with ID starting with ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31 not found: ID does not exist" containerID="ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31"
Sep 10 22:21:51.685706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.684795    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31"} err="failed to get container status \"ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31\": rpc error: code = NotFound desc = could not find container \"ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31\": container with ID starting with ce89529737537d28c7c387071fa7afaa49ab8b3e8441eba93203430edec9ec31 not found: ID does not exist"
Sep 10 22:21:51.687705 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.687656    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-44dc54-openshift-default-6fcb6c6c66-bhtgd"]
Sep 10 22:21:51.901905 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.901780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"]
Sep 10 22:21:51.904631 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:21:51.904607    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d991a2_b436_48b8_99de_9e47900514ba.slice/crio-6622daf8dfe972e8c44327afa33afdc7ccc33e89da5ec510bc9da16cadd90a01 WatchSource:0}: Error finding container 6622daf8dfe972e8c44327afa33afdc7ccc33e89da5ec510bc9da16cadd90a01: Status 404 returned error can't find the container with id 6622daf8dfe972e8c44327afa33afdc7ccc33e89da5ec510bc9da16cadd90a01
Sep 10 22:21:51.949691 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:51.949489    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"]
Sep 10 22:21:51.956891 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:21:51.956861    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c88942_ba0d_4e25_a02e_be37d11fc14e.slice/crio-daaf0db620c6bbb35533e97cecd648f60a91856bd89564efde98be74a2822cb5 WatchSource:0}: Error finding container daaf0db620c6bbb35533e97cecd648f60a91856bd89564efde98be74a2822cb5: Status 404 returned error can't find the container with id daaf0db620c6bbb35533e97cecd648f60a91856bd89564efde98be74a2822cb5
Sep 10 22:21:52.662361 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.662325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm" event={"ID":"88c88942-ba0d-4e25-a02e-be37d11fc14e","Type":"ContainerStarted","Data":"17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a"}
Sep 10 22:21:52.662829 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.662388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:21:52.662829 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.662412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm" event={"ID":"88c88942-ba0d-4e25-a02e-be37d11fc14e","Type":"ContainerStarted","Data":"daaf0db620c6bbb35533e97cecd648f60a91856bd89564efde98be74a2822cb5"}
Sep 10 22:21:52.663890 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.663868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw" event={"ID":"a8d991a2-b436-48b8-99de-9e47900514ba","Type":"ContainerStarted","Data":"9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772"}
Sep 10 22:21:52.664008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.663905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw" event={"ID":"a8d991a2-b436-48b8-99de-9e47900514ba","Type":"ContainerStarted","Data":"6622daf8dfe972e8c44327afa33afdc7ccc33e89da5ec510bc9da16cadd90a01"}
Sep 10 22:21:52.664008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.663935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:21:52.679784 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.679737    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm" podStartSLOduration=1.679719215 podStartE2EDuration="1.679719215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:21:52.677991345 +0000 UTC m=+982.062496771" watchObservedRunningTime="2026-09-10 22:21:52.679719215 +0000 UTC m=+982.064224644"
Sep 10 22:21:52.694243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:52.694179    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw" podStartSLOduration=1.694165966 podStartE2EDuration="1.694165966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:21:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:21:52.693740387 +0000 UTC m=+982.078245821" watchObservedRunningTime="2026-09-10 22:21:52.694165966 +0000 UTC m=+982.078671388"
Sep 10 22:21:53.110744 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:53.110712    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="129e09c3-9ae9-4c39-9493-ea5a2654b5aa" path="/var/lib/kubelet/pods/129e09c3-9ae9-4c39-9493-ea5a2654b5aa/volumes"
Sep 10 22:21:56.823426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:21:56.823391    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:22:03.670943 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:03.670915    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:22:03.671359 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:03.670992    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:22:04.311816 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.311780    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"]
Sep 10 22:22:04.315611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.315592    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.317848 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.317828    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c3af05\"/\"kube-root-ca.crt\""
Sep 10 22:22:04.318499 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.318480    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c3af05\"/\"openshift-service-ca.crt\""
Sep 10 22:22:04.318592 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.318566    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c3af05\"/\"default-dockercfg-kcszp\""
Sep 10 22:22:04.318652 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.318608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c3af05\"/\"test-model-c3af05-kserve-self-signed-certs\""
Sep 10 22:22:04.323974 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.323952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"]
Sep 10 22:22:04.410402 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.410345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-dshm\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.410402 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.410396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5zkq\" (UniqueName: \"kubernetes.io/projected/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-kube-api-access-x5zkq\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.410694 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.410476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tmp-dir\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.410694 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.410606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-model-cache\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.410694 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.410654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tls-certs\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.410874 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.410711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-home\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511138 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511105    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-dshm\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511138 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511143    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5zkq\" (UniqueName: \"kubernetes.io/projected/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-kube-api-access-x5zkq\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511335 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511174    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tmp-dir\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511335 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-model-cache\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511335 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tls-certs\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511335 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-home\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511627    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tmp-dir\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-model-cache\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.511809 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.511663    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-home\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.513648 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.513624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-dshm\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.513806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.513787    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tls-certs\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.519314 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.519294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5zkq\" (UniqueName: \"kubernetes.io/projected/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-kube-api-access-x5zkq\") pod \"test-model-c3af05-kserve-54b8868449-7v65d\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") " pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.627432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.627351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:04.899980 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:04.899956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"]
Sep 10 22:22:04.902116 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:22:04.902088    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod967f00cd_9eb0_4eaa_92c6_111c2d2fe826.slice/crio-328f75d4f9856e2db5e9c9578bc4206e0f12db93449b5f9c1133c3b69378928b WatchSource:0}: Error finding container 328f75d4f9856e2db5e9c9578bc4206e0f12db93449b5f9c1133c3b69378928b: Status 404 returned error can't find the container with id 328f75d4f9856e2db5e9c9578bc4206e0f12db93449b5f9c1133c3b69378928b
Sep 10 22:22:05.732042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:05.732009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d" event={"ID":"967f00cd-9eb0-4eaa-92c6-111c2d2fe826","Type":"ContainerStarted","Data":"556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24"}
Sep 10 22:22:05.732212 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:05.732051    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d" event={"ID":"967f00cd-9eb0-4eaa-92c6-111c2d2fe826","Type":"ContainerStarted","Data":"328f75d4f9856e2db5e9c9578bc4206e0f12db93449b5f9c1133c3b69378928b"}
Sep 10 22:22:05.749248 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:05.749203    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d" podStartSLOduration=1.749188228 podStartE2EDuration="1.749188228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:22:05.746995307 +0000 UTC m=+995.131500745" watchObservedRunningTime="2026-09-10 22:22:05.749188228 +0000 UTC m=+995.133693649"
Sep 10 22:22:13.803263 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:13.803223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:22:14.628022 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:14.627984    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:14.628243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:14.628036    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:14.640459 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:14.640407    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:14.787677 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:14.787650    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:22:29.404697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:29.404658    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:22:44.206627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.206592    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"]
Sep 10 22:22:44.212021 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.211994    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.214946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.214916    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-65b791\"/\"default-dockercfg-xjjxs\""
Sep 10 22:22:44.214946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.214937    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-65b791\"/\"openshift-service-ca.crt\""
Sep 10 22:22:44.215134 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.214954    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-65b791\"/\"explicit-model-65b791-kserve-self-signed-certs\""
Sep 10 22:22:44.215134 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.214916    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-65b791\"/\"kube-root-ca.crt\""
Sep 10 22:22:44.218364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.218343    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"]
Sep 10 22:22:44.271027 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.271000    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-home\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.271170 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.271045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8ecee7-c30e-433e-911c-49988d110091-tls-certs\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.271170 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.271099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-tmp-dir\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.271170 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.271136    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-model-cache\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.271170 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.271167    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq86d\" (UniqueName: \"kubernetes.io/projected/aa8ecee7-c30e-433e-911c-49988d110091-kube-api-access-fq86d\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.271375 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.271193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-dshm\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-home\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8ecee7-c30e-433e-911c-49988d110091-tls-certs\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-tmp-dir\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372359 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-model-cache\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372359 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq86d\" (UniqueName: \"kubernetes.io/projected/aa8ecee7-c30e-433e-911c-49988d110091-kube-api-access-fq86d\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372359 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-dshm\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-tmp-dir\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372544    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-home\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.372692 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.372649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-model-cache\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.374545 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.374523    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-dshm\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.374913 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.374893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8ecee7-c30e-433e-911c-49988d110091-tls-certs\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.380071 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.380049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq86d\" (UniqueName: \"kubernetes.io/projected/aa8ecee7-c30e-433e-911c-49988d110091-kube-api-access-fq86d\") pod \"explicit-model-65b791-kserve-cc84d5fbf-7lwbk\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") " pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.523853 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.523823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:44.795802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.795731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"]
Sep 10 22:22:44.798943 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:22:44.798915    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa8ecee7_c30e_433e_911c_49988d110091.slice/crio-f8faae296c1f2913fd445ee2e2049f05b7d66a429bf0357e227768048d79c48b WatchSource:0}: Error finding container f8faae296c1f2913fd445ee2e2049f05b7d66a429bf0357e227768048d79c48b: Status 404 returned error can't find the container with id f8faae296c1f2913fd445ee2e2049f05b7d66a429bf0357e227768048d79c48b
Sep 10 22:22:44.922040 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.922007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk" event={"ID":"aa8ecee7-c30e-433e-911c-49988d110091","Type":"ContainerStarted","Data":"6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147"}
Sep 10 22:22:44.922040 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.922047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk" event={"ID":"aa8ecee7-c30e-433e-911c-49988d110091","Type":"ContainerStarted","Data":"f8faae296c1f2913fd445ee2e2049f05b7d66a429bf0357e227768048d79c48b"}
Sep 10 22:22:44.940646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:44.940595    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk" podStartSLOduration=0.940581616 podStartE2EDuration="940.581616ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:22:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:22:44.938704282 +0000 UTC m=+1034.323209707" watchObservedRunningTime="2026-09-10 22:22:44.940581616 +0000 UTC m=+1034.325087040"
Sep 10 22:22:46.635363 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:46.635330    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"]
Sep 10 22:22:47.519025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:47.518989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:22:54.524517 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:54.524481    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:22:57.305575 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.305539    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"]
Sep 10 22:22:57.305999 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.305734    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm" podUID="88c88942-ba0d-4e25-a02e-be37d11fc14e" containerName="payload-processing" containerID="cri-o://17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a" gracePeriod=30
Sep 10 22:22:57.311629 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.311602    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"]
Sep 10 22:22:57.311871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.311810    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw" podUID="a8d991a2-b436-48b8-99de-9e47900514ba" containerName="payload-pre-processing" containerID="cri-o://9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772" gracePeriod=30
Sep 10 22:22:57.583949 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.583929    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:22:57.586603 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.586571    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:22:57.696659 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.696622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c88942-ba0d-4e25-a02e-be37d11fc14e-plugins-config-volume\") pod \"88c88942-ba0d-4e25-a02e-be37d11fc14e\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") "
Sep 10 22:22:57.696846 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.696733    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mh2jt\" (UniqueName: \"kubernetes.io/projected/a8d991a2-b436-48b8-99de-9e47900514ba-kube-api-access-mh2jt\") pod \"a8d991a2-b436-48b8-99de-9e47900514ba\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") "
Sep 10 22:22:57.696846 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.696754    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d991a2-b436-48b8-99de-9e47900514ba-plugins-config-volume\") pod \"a8d991a2-b436-48b8-99de-9e47900514ba\" (UID: \"a8d991a2-b436-48b8-99de-9e47900514ba\") "
Sep 10 22:22:57.696846 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.696781    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hq5gh\" (UniqueName: \"kubernetes.io/projected/88c88942-ba0d-4e25-a02e-be37d11fc14e-kube-api-access-hq5gh\") pod \"88c88942-ba0d-4e25-a02e-be37d11fc14e\" (UID: \"88c88942-ba0d-4e25-a02e-be37d11fc14e\") "
Sep 10 22:22:57.697107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.697075    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88c88942-ba0d-4e25-a02e-be37d11fc14e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88c88942-ba0d-4e25-a02e-be37d11fc14e" (UID: "88c88942-ba0d-4e25-a02e-be37d11fc14e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:22:57.697192 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.697164    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8d991a2-b436-48b8-99de-9e47900514ba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a8d991a2-b436-48b8-99de-9e47900514ba" (UID: "a8d991a2-b436-48b8-99de-9e47900514ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:22:57.699078 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.699054    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d991a2-b436-48b8-99de-9e47900514ba-kube-api-access-mh2jt" (OuterVolumeSpecName: "kube-api-access-mh2jt") pod "a8d991a2-b436-48b8-99de-9e47900514ba" (UID: "a8d991a2-b436-48b8-99de-9e47900514ba"). InnerVolumeSpecName "kube-api-access-mh2jt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:22:57.699204 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.699073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88c88942-ba0d-4e25-a02e-be37d11fc14e-kube-api-access-hq5gh" (OuterVolumeSpecName: "kube-api-access-hq5gh") pod "88c88942-ba0d-4e25-a02e-be37d11fc14e" (UID: "88c88942-ba0d-4e25-a02e-be37d11fc14e"). InnerVolumeSpecName "kube-api-access-hq5gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:22:57.797747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.797723    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hq5gh\" (UniqueName: \"kubernetes.io/projected/88c88942-ba0d-4e25-a02e-be37d11fc14e-kube-api-access-hq5gh\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:22:57.797747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.797746    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88c88942-ba0d-4e25-a02e-be37d11fc14e-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:22:57.797891 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.797755    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh2jt\" (UniqueName: \"kubernetes.io/projected/a8d991a2-b436-48b8-99de-9e47900514ba-kube-api-access-mh2jt\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:22:57.797891 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.797765    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a8d991a2-b436-48b8-99de-9e47900514ba-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:22:57.997565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.997528    2563 generic.go:356] "Generic (PLEG): container finished" podID="88c88942-ba0d-4e25-a02e-be37d11fc14e" containerID="17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a" exitCode=0
Sep 10 22:22:57.997762 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.997588    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"
Sep 10 22:22:57.997762 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.997616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm" event={"ID":"88c88942-ba0d-4e25-a02e-be37d11fc14e","Type":"ContainerDied","Data":"17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a"}
Sep 10 22:22:57.997762 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.997707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm" event={"ID":"88c88942-ba0d-4e25-a02e-be37d11fc14e","Type":"ContainerDied","Data":"daaf0db620c6bbb35533e97cecd648f60a91856bd89564efde98be74a2822cb5"}
Sep 10 22:22:57.997762 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.997736    2563 scope.go:117] "RemoveContainer" containerID="17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a"
Sep 10 22:22:57.999391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.999354    2563 generic.go:356] "Generic (PLEG): container finished" podID="a8d991a2-b436-48b8-99de-9e47900514ba" containerID="9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772" exitCode=0
Sep 10 22:22:57.999528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.999418    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"
Sep 10 22:22:57.999528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.999473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw" event={"ID":"a8d991a2-b436-48b8-99de-9e47900514ba","Type":"ContainerDied","Data":"9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772"}
Sep 10 22:22:57.999749 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:57.999527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw" event={"ID":"a8d991a2-b436-48b8-99de-9e47900514ba","Type":"ContainerDied","Data":"6622daf8dfe972e8c44327afa33afdc7ccc33e89da5ec510bc9da16cadd90a01"}
Sep 10 22:22:58.008932 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.008916    2563 scope.go:117] "RemoveContainer" containerID="17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a"
Sep 10 22:22:58.009231 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:22:58.009212    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a\": container with ID starting with 17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a not found: ID does not exist" containerID="17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a"
Sep 10 22:22:58.009291 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.009240    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a"} err="failed to get container status \"17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a\": rpc error: code = NotFound desc = could not find container \"17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a\": container with ID starting with 17d93ca4e060134e73b1047daf637e88422d359249fc5146fbec39d6cd16fe0a not found: ID does not exist"
Sep 10 22:22:58.009291 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.009257    2563 scope.go:117] "RemoveContainer" containerID="9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772"
Sep 10 22:22:58.019993 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.019976    2563 scope.go:117] "RemoveContainer" containerID="9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772"
Sep 10 22:22:58.020296 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:22:58.020276    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772\": container with ID starting with 9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772 not found: ID does not exist" containerID="9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772"
Sep 10 22:22:58.020348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.020302    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772"} err="failed to get container status \"9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772\": rpc error: code = NotFound desc = could not find container \"9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772\": container with ID starting with 9066e4fe40980e5e81497b66fa318dc67878170681e2470d3e1a865365a2a772 not found: ID does not exist"
Sep 10 22:22:58.023146 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.023124    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"]
Sep 10 22:22:58.029742 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.029721    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-65b791-6b9ff7c557-jt4zm"]
Sep 10 22:22:58.039994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.039974    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"]
Sep 10 22:22:58.046917 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:58.046895    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-65b791-6597b44d86-kdmnw"]
Sep 10 22:22:59.111056 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:59.111019    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88c88942-ba0d-4e25-a02e-be37d11fc14e" path="/var/lib/kubelet/pods/88c88942-ba0d-4e25-a02e-be37d11fc14e/volumes"
Sep 10 22:22:59.111585 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:22:59.111546    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d991a2-b436-48b8-99de-9e47900514ba" path="/var/lib/kubelet/pods/a8d991a2-b436-48b8-99de-9e47900514ba/volumes"
Sep 10 22:23:01.491818 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:01.491780    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:23:01.952470 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:01.952351    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk" podUID="aa8ecee7-c30e-433e-911c-49988d110091" containerName="main" containerID="cri-o://6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147" gracePeriod=45
Sep 10 22:23:02.207556 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.207489    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:23:02.338292 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338258    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-home\") pod \"aa8ecee7-c30e-433e-911c-49988d110091\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") "
Sep 10 22:23:02.338487 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338305    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-model-cache\") pod \"aa8ecee7-c30e-433e-911c-49988d110091\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") "
Sep 10 22:23:02.338487 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338331    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8ecee7-c30e-433e-911c-49988d110091-tls-certs\") pod \"aa8ecee7-c30e-433e-911c-49988d110091\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") "
Sep 10 22:23:02.338487 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338415    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq86d\" (UniqueName: \"kubernetes.io/projected/aa8ecee7-c30e-433e-911c-49988d110091-kube-api-access-fq86d\") pod \"aa8ecee7-c30e-433e-911c-49988d110091\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") "
Sep 10 22:23:02.338487 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338430    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-dshm\") pod \"aa8ecee7-c30e-433e-911c-49988d110091\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") "
Sep 10 22:23:02.338708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338615    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-tmp-dir\") pod \"aa8ecee7-c30e-433e-911c-49988d110091\" (UID: \"aa8ecee7-c30e-433e-911c-49988d110091\") "
Sep 10 22:23:02.338708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa8ecee7-c30e-433e-911c-49988d110091" (UID: "aa8ecee7-c30e-433e-911c-49988d110091"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:02.338708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338652    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-home" (OuterVolumeSpecName: "home") pod "aa8ecee7-c30e-433e-911c-49988d110091" (UID: "aa8ecee7-c30e-433e-911c-49988d110091"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:02.338839 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338820    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa8ecee7-c30e-433e-911c-49988d110091" (UID: "aa8ecee7-c30e-433e-911c-49988d110091"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:02.339015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.338997    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:02.339015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.339014    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:02.339306 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.339023    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:02.340849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.340823    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-dshm" (OuterVolumeSpecName: "dshm") pod "aa8ecee7-c30e-433e-911c-49988d110091" (UID: "aa8ecee7-c30e-433e-911c-49988d110091"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:02.340849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.340837    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa8ecee7-c30e-433e-911c-49988d110091-kube-api-access-fq86d" (OuterVolumeSpecName: "kube-api-access-fq86d") pod "aa8ecee7-c30e-433e-911c-49988d110091" (UID: "aa8ecee7-c30e-433e-911c-49988d110091"). InnerVolumeSpecName "kube-api-access-fq86d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:23:02.340968 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.340850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa8ecee7-c30e-433e-911c-49988d110091-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa8ecee7-c30e-433e-911c-49988d110091" (UID: "aa8ecee7-c30e-433e-911c-49988d110091"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:23:02.440046 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.440008    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq86d\" (UniqueName: \"kubernetes.io/projected/aa8ecee7-c30e-433e-911c-49988d110091-kube-api-access-fq86d\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:02.440046 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.440042    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa8ecee7-c30e-433e-911c-49988d110091-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:02.440225 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:02.440056    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa8ecee7-c30e-433e-911c-49988d110091-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:03.028779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.028738    2563 generic.go:356] "Generic (PLEG): container finished" podID="aa8ecee7-c30e-433e-911c-49988d110091" containerID="6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147" exitCode=0
Sep 10 22:23:03.029232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.028799    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"
Sep 10 22:23:03.029232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.028820    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk" event={"ID":"aa8ecee7-c30e-433e-911c-49988d110091","Type":"ContainerDied","Data":"6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147"}
Sep 10 22:23:03.029232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.028860    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk" event={"ID":"aa8ecee7-c30e-433e-911c-49988d110091","Type":"ContainerDied","Data":"f8faae296c1f2913fd445ee2e2049f05b7d66a429bf0357e227768048d79c48b"}
Sep 10 22:23:03.029232 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.028883    2563 scope.go:117] "RemoveContainer" containerID="6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147"
Sep 10 22:23:03.038558 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.038537    2563 scope.go:117] "RemoveContainer" containerID="6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147"
Sep 10 22:23:03.038907 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:23:03.038869    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147\": container with ID starting with 6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147 not found: ID does not exist" containerID="6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147"
Sep 10 22:23:03.039019 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.038906    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147"} err="failed to get container status \"6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147\": rpc error: code = NotFound desc = could not find container \"6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147\": container with ID starting with 6d11c1637aa45d1be8c408691c1beed4884d31510d56474f8dd46495b67e7147 not found: ID does not exist"
Sep 10 22:23:03.050387 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.050359    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"]
Sep 10 22:23:03.053579 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.053558    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-65b791/explicit-model-65b791-kserve-cc84d5fbf-7lwbk"]
Sep 10 22:23:03.110517 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:03.110491    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa8ecee7-c30e-433e-911c-49988d110091" path="/var/lib/kubelet/pods/aa8ecee7-c30e-433e-911c-49988d110091/volumes"
Sep 10 22:23:05.691528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.691497    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"]
Sep 10 22:23:05.692018 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.691998    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d991a2-b436-48b8-99de-9e47900514ba" containerName="payload-pre-processing"
Sep 10 22:23:05.692103 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692022    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d991a2-b436-48b8-99de-9e47900514ba" containerName="payload-pre-processing"
Sep 10 22:23:05.692103 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692035    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88c88942-ba0d-4e25-a02e-be37d11fc14e" containerName="payload-processing"
Sep 10 22:23:05.692103 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692043    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="88c88942-ba0d-4e25-a02e-be37d11fc14e" containerName="payload-processing"
Sep 10 22:23:05.692103 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692069    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa8ecee7-c30e-433e-911c-49988d110091" containerName="main"
Sep 10 22:23:05.692103 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692078    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa8ecee7-c30e-433e-911c-49988d110091" containerName="main"
Sep 10 22:23:05.692348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692160    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="88c88942-ba0d-4e25-a02e-be37d11fc14e" containerName="payload-processing"
Sep 10 22:23:05.692348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692178    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d991a2-b436-48b8-99de-9e47900514ba" containerName="payload-pre-processing"
Sep 10 22:23:05.692348 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.692188    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa8ecee7-c30e-433e-911c-49988d110091" containerName="main"
Sep 10 22:23:05.696479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.696457    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.699320 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.699298    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-c78ff4-openshift-default-dockercfg-64rfm\""
Sep 10 22:23:05.706197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.706170    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"]
Sep 10 22:23:05.767718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.767693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-socket\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.767860 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.767722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22cdae30-2c88-4431-ac98-c9275b653c43-istio-podinfo\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.767860 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.767757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22cdae30-2c88-4431-ac98-c9275b653c43-istiod-ca-cert\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.767860 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.767835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-certs\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.768007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.767871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-data\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.768007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.767891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-envoy\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.768007 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.767985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4zx\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-kube-api-access-sm4zx\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.768133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.768023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-istio-token\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.768133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.768121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-credential-socket\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869605 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4zx\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-kube-api-access-sm4zx\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869605 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-istio-token\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-credential-socket\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-socket\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22cdae30-2c88-4431-ac98-c9275b653c43-istio-podinfo\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869726    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22cdae30-2c88-4431-ac98-c9275b653c43-istiod-ca-cert\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-certs\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-data\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.869836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.869815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-envoy\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.870160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.870127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-credential-socket\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.870242 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.870212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-socket\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.870295 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.870268    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-certs\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.870519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.870495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-data\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.870638 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.870567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22cdae30-2c88-4431-ac98-c9275b653c43-istiod-ca-cert\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.872340 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.872321    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-envoy\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.872480 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.872461    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22cdae30-2c88-4431-ac98-c9275b653c43-istio-podinfo\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.877909 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.877891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-istio-token\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:05.878247 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:05.878226    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4zx\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-kube-api-access-sm4zx\") pod \"e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") " pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:06.007715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:06.007688    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:06.279836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:06.279810    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"]
Sep 10 22:23:06.282167 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:23:06.282127    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22cdae30_2c88_4431_ac98_c9275b653c43.slice/crio-91fe23b51891f46be79f5f97d315b44284439ec219d01fcf961014ae9ef09b1c WatchSource:0}: Error finding container 91fe23b51891f46be79f5f97d315b44284439ec219d01fcf961014ae9ef09b1c: Status 404 returned error can't find the container with id 91fe23b51891f46be79f5f97d315b44284439ec219d01fcf961014ae9ef09b1c
Sep 10 22:23:06.284657 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:06.284624    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:23:06.284745 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:06.284687    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:23:06.284745 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:06.284712    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:23:07.052164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:07.052127    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf" event={"ID":"22cdae30-2c88-4431-ac98-c9275b653c43","Type":"ContainerStarted","Data":"f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0"}
Sep 10 22:23:07.052578 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:07.052180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf" event={"ID":"22cdae30-2c88-4431-ac98-c9275b653c43","Type":"ContainerStarted","Data":"91fe23b51891f46be79f5f97d315b44284439ec219d01fcf961014ae9ef09b1c"}
Sep 10 22:23:07.092335 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:07.092282    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf" podStartSLOduration=2.092266956 podStartE2EDuration="2.092266956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:07.090779187 +0000 UTC m=+1056.475284612" watchObservedRunningTime="2026-09-10 22:23:07.092266956 +0000 UTC m=+1056.476772381"
Sep 10 22:23:08.008631 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:08.008596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:08.013427 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:08.013402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:08.056569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:08.056541    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:08.057471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:08.057452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:14.108377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.108344    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"]
Sep 10 22:23:14.111798 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.111780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.114733 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.114710    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-c78ff4\"/\"kube-root-ca.crt\""
Sep 10 22:23:14.114854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.114707    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-c78ff4\"/\"openshift-service-ca.crt\""
Sep 10 22:23:14.114854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.114790    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-c78ff4\"/\"noten-model-c78ff4-kserve-self-signed-certs\""
Sep 10 22:23:14.114854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.114840    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-c78ff4\"/\"default-dockercfg-9n2k8\""
Sep 10 22:23:14.120702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.120675    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"]
Sep 10 22:23:14.247846 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.247808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-home\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.248042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.247858    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ae8b0c-37ef-466c-b9c8-1acced67691e-tls-certs\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.248042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.247879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-model-cache\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.248042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.248017    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-dshm\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.248198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.248094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b25j\" (UniqueName: \"kubernetes.io/projected/27ae8b0c-37ef-466c-b9c8-1acced67691e-kube-api-access-4b25j\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.248198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.248179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-tmp-dir\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.348870 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.348819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b25j\" (UniqueName: \"kubernetes.io/projected/27ae8b0c-37ef-466c-b9c8-1acced67691e-kube-api-access-4b25j\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.348870 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.348883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-tmp-dir\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.349128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.348908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-home\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.349128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.348938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ae8b0c-37ef-466c-b9c8-1acced67691e-tls-certs\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.349128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.348962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-model-cache\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.349128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.349024    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-dshm\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.349323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.349302    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-tmp-dir\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.349389 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.349369    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-home\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.349509 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.349491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-model-cache\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.351428 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.351403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-dshm\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.351680 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.351665    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ae8b0c-37ef-466c-b9c8-1acced67691e-tls-certs\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.356494 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.356472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b25j\" (UniqueName: \"kubernetes.io/projected/27ae8b0c-37ef-466c-b9c8-1acced67691e-kube-api-access-4b25j\") pod \"noten-model-c78ff4-kserve-666676744c-8sffx\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") " pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.421630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.421572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:14.697364 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:14.697339    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"]
Sep 10 22:23:14.699486 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:23:14.699455    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27ae8b0c_37ef_466c_b9c8_1acced67691e.slice/crio-032187cc2bbcfe828935653ca7b9be106019dfb791ec16aa48d0cef9c103497f WatchSource:0}: Error finding container 032187cc2bbcfe828935653ca7b9be106019dfb791ec16aa48d0cef9c103497f: Status 404 returned error can't find the container with id 032187cc2bbcfe828935653ca7b9be106019dfb791ec16aa48d0cef9c103497f
Sep 10 22:23:15.091526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:15.091487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx" event={"ID":"27ae8b0c-37ef-466c-b9c8-1acced67691e","Type":"ContainerStarted","Data":"353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940"}
Sep 10 22:23:15.091697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:15.091536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx" event={"ID":"27ae8b0c-37ef-466c-b9c8-1acced67691e","Type":"ContainerStarted","Data":"032187cc2bbcfe828935653ca7b9be106019dfb791ec16aa48d0cef9c103497f"}
Sep 10 22:23:15.108227 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:15.108177    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx" podStartSLOduration=1.108158458 podStartE2EDuration="1.108158458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:15.107035382 +0000 UTC m=+1064.491540804" watchObservedRunningTime="2026-09-10 22:23:15.108158458 +0000 UTC m=+1064.492663884"
Sep 10 22:23:23.941939 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:23.941861    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"]
Sep 10 22:23:23.945249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:23.945232    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:23.947689 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:23.947667    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b2c924c3-dockercfg-kkfxf\""
Sep 10 22:23:23.947805 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:23.947691    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b2c924c3\""
Sep 10 22:23:23.956758 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:23.956736    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"]
Sep 10 22:23:24.038987 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.038947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8psfl\" (UniqueName: \"kubernetes.io/projected/b6803c49-359c-42ed-8539-6c5ce2620473-kube-api-access-8psfl\") pod \"payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") " pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:24.039152 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.039005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6803c49-359c-42ed-8539-6c5ce2620473-plugins-config-volume\") pod \"payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") " pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:24.140263 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.140220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8psfl\" (UniqueName: \"kubernetes.io/projected/b6803c49-359c-42ed-8539-6c5ce2620473-kube-api-access-8psfl\") pod \"payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") " pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:24.140484 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.140312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6803c49-359c-42ed-8539-6c5ce2620473-plugins-config-volume\") pod \"payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") " pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:24.141034 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.141013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6803c49-359c-42ed-8539-6c5ce2620473-plugins-config-volume\") pod \"payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") " pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:24.149020 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.148992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8psfl\" (UniqueName: \"kubernetes.io/projected/b6803c49-359c-42ed-8539-6c5ce2620473-kube-api-access-8psfl\") pod \"payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") " pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:24.256254 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.256215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:24.421972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.421934    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:24.422171 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.421985    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:24.434377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.434355    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:24.527614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:24.527586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"]
Sep 10 22:23:24.529805 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:23:24.529778    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6803c49_359c_42ed_8539_6c5ce2620473.slice/crio-145f23e78efa4ec794705c4c84270d702ff19bd731ff54ee668d4e6d8c44a886 WatchSource:0}: Error finding container 145f23e78efa4ec794705c4c84270d702ff19bd731ff54ee668d4e6d8c44a886: Status 404 returned error can't find the container with id 145f23e78efa4ec794705c4c84270d702ff19bd731ff54ee668d4e6d8c44a886
Sep 10 22:23:25.141492 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:25.141453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w" event={"ID":"b6803c49-359c-42ed-8539-6c5ce2620473","Type":"ContainerStarted","Data":"2e4b7c525d225949898dac215a4d8ff2134e6d2f57a62dd011c0b84e1a2447f9"}
Sep 10 22:23:25.141893 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:25.141505    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w" event={"ID":"b6803c49-359c-42ed-8539-6c5ce2620473","Type":"ContainerStarted","Data":"145f23e78efa4ec794705c4c84270d702ff19bd731ff54ee668d4e6d8c44a886"}
Sep 10 22:23:25.141893 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:25.141531    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:25.153395 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:25.153370    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:25.157917 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:25.157876    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w" podStartSLOduration=2.157862512 podStartE2EDuration="2.157862512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:25.155792313 +0000 UTC m=+1074.540297752" watchObservedRunningTime="2026-09-10 22:23:25.157862512 +0000 UTC m=+1074.542367936"
Sep 10 22:23:29.344688 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:29.344655    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:23:35.695616 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:35.695586    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"]
Sep 10 22:23:36.149279 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:36.149251    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:23:36.699448 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:36.699398    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"]
Sep 10 22:23:36.794128 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:36.794096    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"]
Sep 10 22:23:36.794367 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:36.794326    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf" podUID="22cdae30-2c88-4431-ac98-c9275b653c43" containerName="istio-proxy" containerID="cri-o://f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0" gracePeriod=30
Sep 10 22:23:37.883502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:37.883462    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"]
Sep 10 22:23:38.058854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:38.058818    2563 patch_prober.go:28] interesting pod/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 22:23:38.059008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:38.058903    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf" podUID="22cdae30-2c88-4431-ac98-c9275b653c43" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 22:23:42.048812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.048789    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:42.105759 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105692    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-envoy\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.105759 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22cdae30-2c88-4431-ac98-c9275b653c43-istiod-ca-cert\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.105759 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105751    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-certs\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.106015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-data\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.106015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105814    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-istio-token\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.106015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-socket\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.106015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105861    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm4zx\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-kube-api-access-sm4zx\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.106015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105910    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-credential-socket\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.106015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.105941    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22cdae30-2c88-4431-ac98-c9275b653c43-istio-podinfo\") pod \"22cdae30-2c88-4431-ac98-c9275b653c43\" (UID: \"22cdae30-2c88-4431-ac98-c9275b653c43\") "
Sep 10 22:23:42.106342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106083    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:42.106342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106155    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-data" (OuterVolumeSpecName: "istio-data") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:42.106342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22cdae30-2c88-4431-ac98-c9275b653c43-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:23:42.106342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106315    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:42.106558 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106373    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.106558 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106398    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22cdae30-2c88-4431-ac98-c9275b653c43-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.106558 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106413    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.106558 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.106457    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:42.108156 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.108135    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:42.108266 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.108198    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-kube-api-access-sm4zx" (OuterVolumeSpecName: "kube-api-access-sm4zx") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "kube-api-access-sm4zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:23:42.108725 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.108707    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22cdae30-2c88-4431-ac98-c9275b653c43-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:23:42.108725 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.108708    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-istio-token" (OuterVolumeSpecName: "istio-token") pod "22cdae30-2c88-4431-ac98-c9275b653c43" (UID: "22cdae30-2c88-4431-ac98-c9275b653c43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:23:42.207156 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.207117    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.207156 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.207151    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.207346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.207165    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm4zx\" (UniqueName: \"kubernetes.io/projected/22cdae30-2c88-4431-ac98-c9275b653c43-kube-api-access-sm4zx\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.207346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.207180    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.207346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.207193    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22cdae30-2c88-4431-ac98-c9275b653c43-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.207346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.207205    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22cdae30-2c88-4431-ac98-c9275b653c43-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:42.224323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.224285    2563 generic.go:356] "Generic (PLEG): container finished" podID="22cdae30-2c88-4431-ac98-c9275b653c43" containerID="f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0" exitCode=0
Sep 10 22:23:42.224493 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.224340    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"
Sep 10 22:23:42.224493 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.224371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf" event={"ID":"22cdae30-2c88-4431-ac98-c9275b653c43","Type":"ContainerDied","Data":"f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0"}
Sep 10 22:23:42.224493 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.224427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf" event={"ID":"22cdae30-2c88-4431-ac98-c9275b653c43","Type":"ContainerDied","Data":"91fe23b51891f46be79f5f97d315b44284439ec219d01fcf961014ae9ef09b1c"}
Sep 10 22:23:42.224493 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.224474    2563 scope.go:117] "RemoveContainer" containerID="f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0"
Sep 10 22:23:42.233707 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.233692    2563 scope.go:117] "RemoveContainer" containerID="f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0"
Sep 10 22:23:42.233999 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:23:42.233982    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0\": container with ID starting with f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0 not found: ID does not exist" containerID="f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0"
Sep 10 22:23:42.234049 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.234007    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0"} err="failed to get container status \"f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0\": rpc error: code = NotFound desc = could not find container \"f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0\": container with ID starting with f4d423d199b5e3807df5c84cfbc15809a6d374088e1ffd770009916804c887e0 not found: ID does not exist"
Sep 10 22:23:42.244911 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.244887    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"]
Sep 10 22:23:42.248578 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.248557    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-c78ff4-openshift-default-6ccfdddcd8-9wnqf"]
Sep 10 22:23:42.698213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:42.698180    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:23:43.110531 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:43.110503    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22cdae30-2c88-4431-ac98-c9275b653c43" path="/var/lib/kubelet/pods/22cdae30-2c88-4431-ac98-c9275b653c43/volumes"
Sep 10 22:23:48.837756 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:48.837718    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"]
Sep 10 22:23:48.838245 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:48.838015    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb" podUID="85021748-c50c-4fa8-ae10-760bb28db67e" containerName="payload-pre-processing" containerID="cri-o://8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a" gracePeriod=30
Sep 10 22:23:49.102298 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.102224    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:23:49.175562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.175530    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42lqk\" (UniqueName: \"kubernetes.io/projected/85021748-c50c-4fa8-ae10-760bb28db67e-kube-api-access-42lqk\") pod \"85021748-c50c-4fa8-ae10-760bb28db67e\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") "
Sep 10 22:23:49.175729 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.175577    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85021748-c50c-4fa8-ae10-760bb28db67e-plugins-config-volume\") pod \"85021748-c50c-4fa8-ae10-760bb28db67e\" (UID: \"85021748-c50c-4fa8-ae10-760bb28db67e\") "
Sep 10 22:23:49.175970 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.175941    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85021748-c50c-4fa8-ae10-760bb28db67e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85021748-c50c-4fa8-ae10-760bb28db67e" (UID: "85021748-c50c-4fa8-ae10-760bb28db67e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:23:49.177857 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.177835    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85021748-c50c-4fa8-ae10-760bb28db67e-kube-api-access-42lqk" (OuterVolumeSpecName: "kube-api-access-42lqk") pod "85021748-c50c-4fa8-ae10-760bb28db67e" (UID: "85021748-c50c-4fa8-ae10-760bb28db67e"). InnerVolumeSpecName "kube-api-access-42lqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:23:49.266509 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.266472    2563 generic.go:356] "Generic (PLEG): container finished" podID="85021748-c50c-4fa8-ae10-760bb28db67e" containerID="8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a" exitCode=0
Sep 10 22:23:49.266658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.266513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb" event={"ID":"85021748-c50c-4fa8-ae10-760bb28db67e","Type":"ContainerDied","Data":"8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a"}
Sep 10 22:23:49.266658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.266521    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"
Sep 10 22:23:49.266658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.266557    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb" event={"ID":"85021748-c50c-4fa8-ae10-760bb28db67e","Type":"ContainerDied","Data":"2a3cad77e3bd978a49c9fa0c4559793cc74b74c202cbb7b3f2988441a9c9ed92"}
Sep 10 22:23:49.266658 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.266579    2563 scope.go:117] "RemoveContainer" containerID="8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a"
Sep 10 22:23:49.275835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.275814    2563 scope.go:117] "RemoveContainer" containerID="8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a"
Sep 10 22:23:49.276106 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:23:49.276075    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a\": container with ID starting with 8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a not found: ID does not exist" containerID="8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a"
Sep 10 22:23:49.276156 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.276115    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a"} err="failed to get container status \"8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a\": rpc error: code = NotFound desc = could not find container \"8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a\": container with ID starting with 8b1da479a685f998816269d5c1091ce9de30c817f846ae102b7bb59f18e6cf9a not found: ID does not exist"
Sep 10 22:23:49.276502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.276486    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42lqk\" (UniqueName: \"kubernetes.io/projected/85021748-c50c-4fa8-ae10-760bb28db67e-kube-api-access-42lqk\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:49.276544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.276504    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85021748-c50c-4fa8-ae10-760bb28db67e-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:49.286730 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.286709    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"]
Sep 10 22:23:49.292714 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:49.292696    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-669e2b-955cb7766-598gb"]
Sep 10 22:23:50.715285 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:50.715237    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf" podUID="c118dc44-4f67-4563-afb0-d14e6fe1487f" containerName="main" containerID="cri-o://4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6" gracePeriod=45
Sep 10 22:23:50.817294 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:50.814957    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:23:50.974355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:50.974274    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:23:51.093323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093287    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-model-cache\") pod \"c118dc44-4f67-4563-afb0-d14e6fe1487f\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") "
Sep 10 22:23:51.093323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093328    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c118dc44-4f67-4563-afb0-d14e6fe1487f-tls-certs\") pod \"c118dc44-4f67-4563-afb0-d14e6fe1487f\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") "
Sep 10 22:23:51.093612 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093363    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sm6bv\" (UniqueName: \"kubernetes.io/projected/c118dc44-4f67-4563-afb0-d14e6fe1487f-kube-api-access-sm6bv\") pod \"c118dc44-4f67-4563-afb0-d14e6fe1487f\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") "
Sep 10 22:23:51.093612 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093388    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-dshm\") pod \"c118dc44-4f67-4563-afb0-d14e6fe1487f\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") "
Sep 10 22:23:51.093612 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093414    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-tmp-dir\") pod \"c118dc44-4f67-4563-afb0-d14e6fe1487f\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") "
Sep 10 22:23:51.093612 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093602    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-home\") pod \"c118dc44-4f67-4563-afb0-d14e6fe1487f\" (UID: \"c118dc44-4f67-4563-afb0-d14e6fe1487f\") "
Sep 10 22:23:51.093815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093679    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-model-cache" (OuterVolumeSpecName: "model-cache") pod "c118dc44-4f67-4563-afb0-d14e6fe1487f" (UID: "c118dc44-4f67-4563-afb0-d14e6fe1487f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:51.093815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093756    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c118dc44-4f67-4563-afb0-d14e6fe1487f" (UID: "c118dc44-4f67-4563-afb0-d14e6fe1487f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:51.093898 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093819    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-home" (OuterVolumeSpecName: "home") pod "c118dc44-4f67-4563-afb0-d14e6fe1487f" (UID: "c118dc44-4f67-4563-afb0-d14e6fe1487f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:51.093953 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093936    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:51.093995 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093961    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:51.093995 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.093976    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:51.095883 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.095863    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c118dc44-4f67-4563-afb0-d14e6fe1487f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c118dc44-4f67-4563-afb0-d14e6fe1487f" (UID: "c118dc44-4f67-4563-afb0-d14e6fe1487f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:23:51.096113 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.096094    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-dshm" (OuterVolumeSpecName: "dshm") pod "c118dc44-4f67-4563-afb0-d14e6fe1487f" (UID: "c118dc44-4f67-4563-afb0-d14e6fe1487f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:51.096186 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.096118    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c118dc44-4f67-4563-afb0-d14e6fe1487f-kube-api-access-sm6bv" (OuterVolumeSpecName: "kube-api-access-sm6bv") pod "c118dc44-4f67-4563-afb0-d14e6fe1487f" (UID: "c118dc44-4f67-4563-afb0-d14e6fe1487f"). InnerVolumeSpecName "kube-api-access-sm6bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:23:51.111599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.111574    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85021748-c50c-4fa8-ae10-760bb28db67e" path="/var/lib/kubelet/pods/85021748-c50c-4fa8-ae10-760bb28db67e/volumes"
Sep 10 22:23:51.195462 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.195421    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c118dc44-4f67-4563-afb0-d14e6fe1487f-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:51.195462 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.195460    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm6bv\" (UniqueName: \"kubernetes.io/projected/c118dc44-4f67-4563-afb0-d14e6fe1487f-kube-api-access-sm6bv\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:51.195462 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.195469    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c118dc44-4f67-4563-afb0-d14e6fe1487f-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:51.280220 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.280184    2563 generic.go:356] "Generic (PLEG): container finished" podID="c118dc44-4f67-4563-afb0-d14e6fe1487f" containerID="4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6" exitCode=0
Sep 10 22:23:51.280406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.280254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf" event={"ID":"c118dc44-4f67-4563-afb0-d14e6fe1487f","Type":"ContainerDied","Data":"4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6"}
Sep 10 22:23:51.280406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.280288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf" event={"ID":"c118dc44-4f67-4563-afb0-d14e6fe1487f","Type":"ContainerDied","Data":"defcbfa75ab09ecb1cbb38faf67918c69ef0386db2465ed50e9fa772b6634f52"}
Sep 10 22:23:51.280406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.280313    2563 scope.go:117] "RemoveContainer" containerID="4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6"
Sep 10 22:23:51.280406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.280312    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"
Sep 10 22:23:51.291479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.291459    2563 scope.go:117] "RemoveContainer" containerID="4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6"
Sep 10 22:23:51.294889 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:23:51.294818    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6\": container with ID starting with 4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6 not found: ID does not exist" containerID="4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6"
Sep 10 22:23:51.294889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.294860    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6"} err="failed to get container status \"4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6\": rpc error: code = NotFound desc = could not find container \"4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6\": container with ID starting with 4b36e94948ea451f6d8a4f557e899a3de37f8890881358e8560ed98c9ca42ee6 not found: ID does not exist"
Sep 10 22:23:51.297074 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.296816    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"]
Sep 10 22:23:51.299371 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.299346    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-669e2b/test-model-669e2b-kserve-6fb6cfb589-2btcf"]
Sep 10 22:23:51.719450 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.719334    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx" podUID="27ae8b0c-37ef-466c-b9c8-1acced67691e" containerName="main" containerID="cri-o://353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940" gracePeriod=45
Sep 10 22:23:51.968042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:51.968017    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:52.104989 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.104957    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-tmp-dir\") pod \"27ae8b0c-37ef-466c-b9c8-1acced67691e\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") "
Sep 10 22:23:52.105198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105010    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b25j\" (UniqueName: \"kubernetes.io/projected/27ae8b0c-37ef-466c-b9c8-1acced67691e-kube-api-access-4b25j\") pod \"27ae8b0c-37ef-466c-b9c8-1acced67691e\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") "
Sep 10 22:23:52.105198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105062    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-model-cache\") pod \"27ae8b0c-37ef-466c-b9c8-1acced67691e\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") "
Sep 10 22:23:52.105198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105086    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-home\") pod \"27ae8b0c-37ef-466c-b9c8-1acced67691e\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") "
Sep 10 22:23:52.105198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105112    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ae8b0c-37ef-466c-b9c8-1acced67691e-tls-certs\") pod \"27ae8b0c-37ef-466c-b9c8-1acced67691e\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") "
Sep 10 22:23:52.105198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-dshm\") pod \"27ae8b0c-37ef-466c-b9c8-1acced67691e\" (UID: \"27ae8b0c-37ef-466c-b9c8-1acced67691e\") "
Sep 10 22:23:52.105488 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105239    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "27ae8b0c-37ef-466c-b9c8-1acced67691e" (UID: "27ae8b0c-37ef-466c-b9c8-1acced67691e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:52.105488 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105383    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-model-cache" (OuterVolumeSpecName: "model-cache") pod "27ae8b0c-37ef-466c-b9c8-1acced67691e" (UID: "27ae8b0c-37ef-466c-b9c8-1acced67691e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:52.105488 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-home" (OuterVolumeSpecName: "home") pod "27ae8b0c-37ef-466c-b9c8-1acced67691e" (UID: "27ae8b0c-37ef-466c-b9c8-1acced67691e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:52.105488 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.105401    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:52.107590 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.107566    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-dshm" (OuterVolumeSpecName: "dshm") pod "27ae8b0c-37ef-466c-b9c8-1acced67691e" (UID: "27ae8b0c-37ef-466c-b9c8-1acced67691e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:52.107694 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.107601    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27ae8b0c-37ef-466c-b9c8-1acced67691e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "27ae8b0c-37ef-466c-b9c8-1acced67691e" (UID: "27ae8b0c-37ef-466c-b9c8-1acced67691e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:23:52.107694 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.107657    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27ae8b0c-37ef-466c-b9c8-1acced67691e-kube-api-access-4b25j" (OuterVolumeSpecName: "kube-api-access-4b25j") pod "27ae8b0c-37ef-466c-b9c8-1acced67691e" (UID: "27ae8b0c-37ef-466c-b9c8-1acced67691e"). InnerVolumeSpecName "kube-api-access-4b25j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:23:52.206625 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.206580    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b25j\" (UniqueName: \"kubernetes.io/projected/27ae8b0c-37ef-466c-b9c8-1acced67691e-kube-api-access-4b25j\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:52.206625 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.206621    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:52.206806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.206635    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:52.206806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.206649    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27ae8b0c-37ef-466c-b9c8-1acced67691e-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:52.206806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.206658    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27ae8b0c-37ef-466c-b9c8-1acced67691e-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:52.287101 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.287066    2563 generic.go:356] "Generic (PLEG): container finished" podID="27ae8b0c-37ef-466c-b9c8-1acced67691e" containerID="353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940" exitCode=0
Sep 10 22:23:52.287237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.287114    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"
Sep 10 22:23:52.287237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.287143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx" event={"ID":"27ae8b0c-37ef-466c-b9c8-1acced67691e","Type":"ContainerDied","Data":"353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940"}
Sep 10 22:23:52.287237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.287185    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx" event={"ID":"27ae8b0c-37ef-466c-b9c8-1acced67691e","Type":"ContainerDied","Data":"032187cc2bbcfe828935653ca7b9be106019dfb791ec16aa48d0cef9c103497f"}
Sep 10 22:23:52.287237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.287201    2563 scope.go:117] "RemoveContainer" containerID="353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940"
Sep 10 22:23:52.295535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.295520    2563 scope.go:117] "RemoveContainer" containerID="353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940"
Sep 10 22:23:52.295789 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:23:52.295771    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940\": container with ID starting with 353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940 not found: ID does not exist" containerID="353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940"
Sep 10 22:23:52.295831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.295799    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940"} err="failed to get container status \"353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940\": rpc error: code = NotFound desc = could not find container \"353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940\": container with ID starting with 353a368799d95075d472e46795133c09e45beb8602d22cbff471d8c967396940 not found: ID does not exist"
Sep 10 22:23:52.306412 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.306389    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"]
Sep 10 22:23:52.312847 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.312827    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-c78ff4/noten-model-c78ff4-kserve-666676744c-8sffx"]
Sep 10 22:23:52.903298 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:52.903261    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d" podUID="967f00cd-9eb0-4eaa-92c6-111c2d2fe826" containerName="main" containerID="cri-o://556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24" gracePeriod=45
Sep 10 22:23:53.112327 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.112292    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27ae8b0c-37ef-466c-b9c8-1acced67691e" path="/var/lib/kubelet/pods/27ae8b0c-37ef-466c-b9c8-1acced67691e/volumes"
Sep 10 22:23:53.112815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.112797    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c118dc44-4f67-4563-afb0-d14e6fe1487f" path="/var/lib/kubelet/pods/c118dc44-4f67-4563-afb0-d14e6fe1487f/volumes"
Sep 10 22:23:53.157531 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.157465    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:23:53.293382 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.293342    2563 generic.go:356] "Generic (PLEG): container finished" podID="967f00cd-9eb0-4eaa-92c6-111c2d2fe826" containerID="556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24" exitCode=0
Sep 10 22:23:53.293382 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.293380    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"
Sep 10 22:23:53.293685 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.293427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d" event={"ID":"967f00cd-9eb0-4eaa-92c6-111c2d2fe826","Type":"ContainerDied","Data":"556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24"}
Sep 10 22:23:53.293685 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.293491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d" event={"ID":"967f00cd-9eb0-4eaa-92c6-111c2d2fe826","Type":"ContainerDied","Data":"328f75d4f9856e2db5e9c9578bc4206e0f12db93449b5f9c1133c3b69378928b"}
Sep 10 22:23:53.293685 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.293511    2563 scope.go:117] "RemoveContainer" containerID="556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24"
Sep 10 22:23:53.301663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.301639    2563 scope.go:117] "RemoveContainer" containerID="556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24"
Sep 10 22:23:53.301935 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:23:53.301918    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24\": container with ID starting with 556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24 not found: ID does not exist" containerID="556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24"
Sep 10 22:23:53.301971 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.301944    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24"} err="failed to get container status \"556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24\": rpc error: code = NotFound desc = could not find container \"556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24\": container with ID starting with 556d57de61561c6a51a886b6d3d6ec8bb898ad927ae21f052101f5cba6e2bd24 not found: ID does not exist"
Sep 10 22:23:53.317233 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317211    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-model-cache\") pod \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") "
Sep 10 22:23:53.317313 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317259    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tmp-dir\") pod \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") "
Sep 10 22:23:53.317313 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317280    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tls-certs\") pod \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") "
Sep 10 22:23:53.317313 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-home\") pod \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") "
Sep 10 22:23:53.317516 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-dshm\") pod \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") "
Sep 10 22:23:53.317516 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317356    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5zkq\" (UniqueName: \"kubernetes.io/projected/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-kube-api-access-x5zkq\") pod \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\" (UID: \"967f00cd-9eb0-4eaa-92c6-111c2d2fe826\") "
Sep 10 22:23:53.317612 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317514    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "967f00cd-9eb0-4eaa-92c6-111c2d2fe826" (UID: "967f00cd-9eb0-4eaa-92c6-111c2d2fe826"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:53.317612 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317536    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-model-cache" (OuterVolumeSpecName: "model-cache") pod "967f00cd-9eb0-4eaa-92c6-111c2d2fe826" (UID: "967f00cd-9eb0-4eaa-92c6-111c2d2fe826"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:53.317715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317610    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-home" (OuterVolumeSpecName: "home") pod "967f00cd-9eb0-4eaa-92c6-111c2d2fe826" (UID: "967f00cd-9eb0-4eaa-92c6-111c2d2fe826"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:53.317794 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317777    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:53.317842 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317801    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:53.317842 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.317814    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:53.319615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.319593    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-dshm" (OuterVolumeSpecName: "dshm") pod "967f00cd-9eb0-4eaa-92c6-111c2d2fe826" (UID: "967f00cd-9eb0-4eaa-92c6-111c2d2fe826"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:23:53.319734 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.319702    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-kube-api-access-x5zkq" (OuterVolumeSpecName: "kube-api-access-x5zkq") pod "967f00cd-9eb0-4eaa-92c6-111c2d2fe826" (UID: "967f00cd-9eb0-4eaa-92c6-111c2d2fe826"). InnerVolumeSpecName "kube-api-access-x5zkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:23:53.319772 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.319753    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "967f00cd-9eb0-4eaa-92c6-111c2d2fe826" (UID: "967f00cd-9eb0-4eaa-92c6-111c2d2fe826"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:23:53.418325 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.418258    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:53.418325 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.418283    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:53.418325 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.418292    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5zkq\" (UniqueName: \"kubernetes.io/projected/967f00cd-9eb0-4eaa-92c6-111c2d2fe826-kube-api-access-x5zkq\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:53.613572 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.613540    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"]
Sep 10 22:23:53.615388 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:53.615365    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c3af05/test-model-c3af05-kserve-54b8868449-7v65d"]
Sep 10 22:23:54.453124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453093    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"]
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453423    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c118dc44-4f67-4563-afb0-d14e6fe1487f" containerName="main"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453449    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c118dc44-4f67-4563-afb0-d14e6fe1487f" containerName="main"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453466    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27ae8b0c-37ef-466c-b9c8-1acced67691e" containerName="main"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453474    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="27ae8b0c-37ef-466c-b9c8-1acced67691e" containerName="main"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453490    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85021748-c50c-4fa8-ae10-760bb28db67e" containerName="payload-pre-processing"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453496    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="85021748-c50c-4fa8-ae10-760bb28db67e" containerName="payload-pre-processing"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453502    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22cdae30-2c88-4431-ac98-c9275b653c43" containerName="istio-proxy"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453507    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22cdae30-2c88-4431-ac98-c9275b653c43" containerName="istio-proxy"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453521    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="967f00cd-9eb0-4eaa-92c6-111c2d2fe826" containerName="main"
Sep 10 22:23:54.453536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453526    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="967f00cd-9eb0-4eaa-92c6-111c2d2fe826" containerName="main"
Sep 10 22:23:54.453871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453580    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="27ae8b0c-37ef-466c-b9c8-1acced67691e" containerName="main"
Sep 10 22:23:54.453871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453590    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c118dc44-4f67-4563-afb0-d14e6fe1487f" containerName="main"
Sep 10 22:23:54.453871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453596    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="85021748-c50c-4fa8-ae10-760bb28db67e" containerName="payload-pre-processing"
Sep 10 22:23:54.453871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453601    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="967f00cd-9eb0-4eaa-92c6-111c2d2fe826" containerName="main"
Sep 10 22:23:54.453871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.453605    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="22cdae30-2c88-4431-ac98-c9275b653c43" containerName="istio-proxy"
Sep 10 22:23:54.457883 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.457866    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"
Sep 10 22:23:54.459845 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.459824    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 22:23:54.459934 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.459881    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlfhj\""
Sep 10 22:23:54.463177 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.463153    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"]
Sep 10 22:23:54.627868 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.627830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ddb34-ef0e-40cb-8834-53578ec3e8f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd\" (UID: \"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"
Sep 10 22:23:54.728496 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.728393    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ddb34-ef0e-40cb-8834-53578ec3e8f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd\" (UID: \"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"
Sep 10 22:23:54.729005 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.728985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ddb34-ef0e-40cb-8834-53578ec3e8f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd\" (UID: \"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"
Sep 10 22:23:54.770421 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:54.770384    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"
Sep 10 22:23:55.037887 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:55.037865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"]
Sep 10 22:23:55.040745 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:23:55.040712    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8ddb34_ef0e_40cb_8834_53578ec3e8f8.slice/crio-e535616c4c138ec618f21fac0600dab04089df98c83713a03de6ffa022ee539c WatchSource:0}: Error finding container e535616c4c138ec618f21fac0600dab04089df98c83713a03de6ffa022ee539c: Status 404 returned error can't find the container with id e535616c4c138ec618f21fac0600dab04089df98c83713a03de6ffa022ee539c
Sep 10 22:23:55.112391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:55.112361    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="967f00cd-9eb0-4eaa-92c6-111c2d2fe826" path="/var/lib/kubelet/pods/967f00cd-9eb0-4eaa-92c6-111c2d2fe826/volumes"
Sep 10 22:23:55.306829 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:55.306737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd" event={"ID":"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8","Type":"ContainerStarted","Data":"4aa38bd61edae8f64884bed0de50a72294d5402120c481a82a51b7b3ffa4b1ba"}
Sep 10 22:23:55.306829 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:55.306785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd" event={"ID":"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8","Type":"ContainerStarted","Data":"e535616c4c138ec618f21fac0600dab04089df98c83713a03de6ffa022ee539c"}
Sep 10 22:23:55.322504 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:55.322431    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd" podStartSLOduration=1.322415759 podStartE2EDuration="1.322415759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:23:55.320910444 +0000 UTC m=+1104.705415869" watchObservedRunningTime="2026-09-10 22:23:55.322415759 +0000 UTC m=+1104.706921183"
Sep 10 22:23:55.667384 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:55.667354    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"]
Sep 10 22:23:55.667770 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:55.667596    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" containerID="cri-o://3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8" gracePeriod=30
Sep 10 22:23:56.313194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:56.313157    2563 generic.go:356] "Generic (PLEG): container finished" podID="fb8ddb34-ef0e-40cb-8834-53578ec3e8f8" containerID="4aa38bd61edae8f64884bed0de50a72294d5402120c481a82a51b7b3ffa4b1ba" exitCode=0
Sep 10 22:23:56.313403 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:56.313241    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd" event={"ID":"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8","Type":"ContainerDied","Data":"4aa38bd61edae8f64884bed0de50a72294d5402120c481a82a51b7b3ffa4b1ba"}
Sep 10 22:23:57.444018 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:57.443984    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"
Sep 10 22:23:57.472475 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:57.472420    2563 patch_prober.go:28] interesting pod/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 22:23:57.472637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:57.472505    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 22:23:57.554244 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:57.554212    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ddb34-ef0e-40cb-8834-53578ec3e8f8-maas-api-service-ca\") pod \"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8\" (UID: \"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8\") "
Sep 10 22:23:57.554595 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:57.554574    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb8ddb34-ef0e-40cb-8834-53578ec3e8f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fb8ddb34-ef0e-40cb-8834-53578ec3e8f8" (UID: "fb8ddb34-ef0e-40cb-8834-53578ec3e8f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:23:57.655926 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:57.655837    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb8ddb34-ef0e-40cb-8834-53578ec3e8f8-maas-api-service-ca\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:23:58.324686 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:58.324657    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"
Sep 10 22:23:58.324858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:58.324695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd" event={"ID":"fb8ddb34-ef0e-40cb-8834-53578ec3e8f8","Type":"ContainerDied","Data":"e535616c4c138ec618f21fac0600dab04089df98c83713a03de6ffa022ee539c"}
Sep 10 22:23:58.324858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:23:58.324741    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e535616c4c138ec618f21fac0600dab04089df98c83713a03de6ffa022ee539c"
Sep 10 22:24:00.924091 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:00.924068    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:24:01.084700 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084674    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56487366-5ec6-41d9-afd0-9534f65f1b3b-istiod-ca-cert\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.084882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-envoy\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.084882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084760    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-data\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.084882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084862    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-credential-socket\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.085032 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084902    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-token\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.085032 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084925    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf58s\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-kube-api-access-hf58s\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.085032 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084954    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-certs\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.085032 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.084990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-data" (OuterVolumeSpecName: "istio-data") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:24:01.085235 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085033    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-socket\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.085235 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085065    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-podinfo\") pod \"56487366-5ec6-41d9-afd0-9534f65f1b3b\" (UID: \"56487366-5ec6-41d9-afd0-9534f65f1b3b\") "
Sep 10 22:24:01.085235 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085142    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56487366-5ec6-41d9-afd0-9534f65f1b3b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:24:01.085386 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085235    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:24:01.085386 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:24:01.085386 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085291    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:24:01.085519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085458    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56487366-5ec6-41d9-afd0-9534f65f1b3b-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.085519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085478    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.085519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085491    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.085519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085505    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.085519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.085517    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.087102 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.087072    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:24:01.087269 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.087254    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-token" (OuterVolumeSpecName: "istio-token") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:24:01.087323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.087271    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:24:01.087362 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.087318    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-kube-api-access-hf58s" (OuterVolumeSpecName: "kube-api-access-hf58s") pod "56487366-5ec6-41d9-afd0-9534f65f1b3b" (UID: "56487366-5ec6-41d9-afd0-9534f65f1b3b"). InnerVolumeSpecName "kube-api-access-hf58s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:24:01.186310 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.186277    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.186310 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.186306    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf58s\" (UniqueName: \"kubernetes.io/projected/56487366-5ec6-41d9-afd0-9534f65f1b3b-kube-api-access-hf58s\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.186522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.186321    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.186522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.186334    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56487366-5ec6-41d9-afd0-9534f65f1b3b-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:01.340603 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.340517    2563 generic.go:356] "Generic (PLEG): container finished" podID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerID="3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8" exitCode=0
Sep 10 22:24:01.340603 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.340571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" event={"ID":"56487366-5ec6-41d9-afd0-9534f65f1b3b","Type":"ContainerDied","Data":"3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8"}
Sep 10 22:24:01.340793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.340610    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc" event={"ID":"56487366-5ec6-41d9-afd0-9534f65f1b3b","Type":"ContainerDied","Data":"97ef8a3efddd27b53ab0f71bf0565ad8240406662c176e71143d302e177893c9"}
Sep 10 22:24:01.340793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.340630    2563 scope.go:117] "RemoveContainer" containerID="3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8"
Sep 10 22:24:01.340793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.340573    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"
Sep 10 22:24:01.352035 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.351866    2563 scope.go:117] "RemoveContainer" containerID="9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12"
Sep 10 22:24:01.356608 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.356585    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"]
Sep 10 22:24:01.360720 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.360649    2563 scope.go:117] "RemoveContainer" containerID="3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8"
Sep 10 22:24:01.361004 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:24:01.360978    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8\": container with ID starting with 3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8 not found: ID does not exist" containerID="3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8"
Sep 10 22:24:01.361098 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.361013    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8"} err="failed to get container status \"3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8\": rpc error: code = NotFound desc = could not find container \"3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8\": container with ID starting with 3a77286d4979756618f077299457669e6835e7de95f8c1b55588b6443f8a36c8 not found: ID does not exist"
Sep 10 22:24:01.361098 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.361041    2563 scope.go:117] "RemoveContainer" containerID="9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12"
Sep 10 22:24:01.361365 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:24:01.361341    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12\": container with ID starting with 9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12 not found: ID does not exist" containerID="9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12"
Sep 10 22:24:01.361488 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.361372    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12"} err="failed to get container status \"9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12\": rpc error: code = NotFound desc = could not find container \"9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12\": container with ID starting with 9bfee720be7f9beb70c92417f72d2a2dfde96ce6ef3d1e1ec6a72b960e98fb12 not found: ID does not exist"
Sep 10 22:24:01.362228 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:01.362207    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-669e2b-openshift-default-545588fdc4-gzkmc"]
Sep 10 22:24:03.110352 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:03.110317    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" path="/var/lib/kubelet/pods/56487366-5ec6-41d9-afd0-9534f65f1b3b/volumes"
Sep 10 22:24:05.152050 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.152015    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"]
Sep 10 22:24:05.152567 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.152226    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w" podUID="b6803c49-359c-42ed-8539-6c5ce2620473" containerName="payload-processing" containerID="cri-o://2e4b7c525d225949898dac215a4d8ff2134e6d2f57a62dd011c0b84e1a2447f9" gracePeriod=30
Sep 10 22:24:05.307843 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.307809    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"]
Sep 10 22:24:05.310334 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.310309    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-b2c924c3-br8fd"]
Sep 10 22:24:05.368987 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.368953    2563 generic.go:356] "Generic (PLEG): container finished" podID="b6803c49-359c-42ed-8539-6c5ce2620473" containerID="2e4b7c525d225949898dac215a4d8ff2134e6d2f57a62dd011c0b84e1a2447f9" exitCode=0
Sep 10 22:24:05.369157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.369031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w" event={"ID":"b6803c49-359c-42ed-8539-6c5ce2620473","Type":"ContainerDied","Data":"2e4b7c525d225949898dac215a4d8ff2134e6d2f57a62dd011c0b84e1a2447f9"}
Sep 10 22:24:05.393061 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.393041    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:24:05.419536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.419478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6803c49-359c-42ed-8539-6c5ce2620473-plugins-config-volume\") pod \"b6803c49-359c-42ed-8539-6c5ce2620473\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") "
Sep 10 22:24:05.419536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.419512    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8psfl\" (UniqueName: \"kubernetes.io/projected/b6803c49-359c-42ed-8539-6c5ce2620473-kube-api-access-8psfl\") pod \"b6803c49-359c-42ed-8539-6c5ce2620473\" (UID: \"b6803c49-359c-42ed-8539-6c5ce2620473\") "
Sep 10 22:24:05.419816 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.419794    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6803c49-359c-42ed-8539-6c5ce2620473-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b6803c49-359c-42ed-8539-6c5ce2620473" (UID: "b6803c49-359c-42ed-8539-6c5ce2620473"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:24:05.421742 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.421710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6803c49-359c-42ed-8539-6c5ce2620473-kube-api-access-8psfl" (OuterVolumeSpecName: "kube-api-access-8psfl") pod "b6803c49-359c-42ed-8539-6c5ce2620473" (UID: "b6803c49-359c-42ed-8539-6c5ce2620473"). InnerVolumeSpecName "kube-api-access-8psfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:24:05.520777 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.520729    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b6803c49-359c-42ed-8539-6c5ce2620473-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:05.520777 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:05.520771    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8psfl\" (UniqueName: \"kubernetes.io/projected/b6803c49-359c-42ed-8539-6c5ce2620473-kube-api-access-8psfl\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:24:06.375078 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:06.375045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w" event={"ID":"b6803c49-359c-42ed-8539-6c5ce2620473","Type":"ContainerDied","Data":"145f23e78efa4ec794705c4c84270d702ff19bd731ff54ee668d4e6d8c44a886"}
Sep 10 22:24:06.375078 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:06.375070    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"
Sep 10 22:24:06.375598 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:06.375106    2563 scope.go:117] "RemoveContainer" containerID="2e4b7c525d225949898dac215a4d8ff2134e6d2f57a62dd011c0b84e1a2447f9"
Sep 10 22:24:06.394838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:06.394813    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"]
Sep 10 22:24:06.398733 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:06.398711    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b2c924c3-65c99c9665-r5g5w"]
Sep 10 22:24:07.111017 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:07.110986    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6803c49-359c-42ed-8539-6c5ce2620473" path="/var/lib/kubelet/pods/b6803c49-359c-42ed-8539-6c5ce2620473/volumes"
Sep 10 22:24:07.111304 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:07.111291    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb8ddb34-ef0e-40cb-8834-53578ec3e8f8" path="/var/lib/kubelet/pods/fb8ddb34-ef0e-40cb-8834-53578ec3e8f8/volumes"
Sep 10 22:24:08.805607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:08.805572    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:24:14.317507 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.317472    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"]
Sep 10 22:24:14.318028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318003    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy"
Sep 10 22:24:14.318028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318027    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy"
Sep 10 22:24:14.318028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318040    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb8ddb34-ef0e-40cb-8834-53578ec3e8f8" containerName="revoke-keys"
Sep 10 22:24:14.318305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318049    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb8ddb34-ef0e-40cb-8834-53578ec3e8f8" containerName="revoke-keys"
Sep 10 22:24:14.318305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318059    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6803c49-359c-42ed-8539-6c5ce2620473" containerName="payload-processing"
Sep 10 22:24:14.318305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318069    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6803c49-359c-42ed-8539-6c5ce2620473" containerName="payload-processing"
Sep 10 22:24:14.318305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318206    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb8ddb34-ef0e-40cb-8834-53578ec3e8f8" containerName="revoke-keys"
Sep 10 22:24:14.318305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318223    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6803c49-359c-42ed-8539-6c5ce2620473" containerName="payload-processing"
Sep 10 22:24:14.318305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318235    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy"
Sep 10 22:24:14.318305 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.318244    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy"
Sep 10 22:24:14.322829 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.322809    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.324875 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.324851    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-e367c1-openshift-default-dockercfg-t4zfh\""
Sep 10 22:24:14.330636 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.330613    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"]
Sep 10 22:24:14.390432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fx7k\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-kube-api-access-9fx7k\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-podinfo\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-data\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-envoy\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390579    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istiod-ca-cert\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-credential-socket\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390797 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-socket\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390797 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-certs\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.390797 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.390758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-token\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.491854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.491823    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-socket\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.491854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.491855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-certs\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.491897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-token\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.491927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fx7k\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-kube-api-access-9fx7k\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.491967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-podinfo\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.491989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-data\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-envoy\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istiod-ca-cert\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-credential-socket\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492506 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-socket\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492506 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492339    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-certs\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492506 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-credential-socket\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492782 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-data\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.492931 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.492815    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istiod-ca-cert\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.494877 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.494852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-podinfo\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.494988 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.494881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-envoy\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.502020 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.501987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-token\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.502259 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.502215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fx7k\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-kube-api-access-9fx7k\") pod \"e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") " pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.633570 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.633484    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:14.912789 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.912764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"]
Sep 10 22:24:14.915132 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:24:14.915105    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc022a573_7bd3_4afb_bb5f_7da6e320fff7.slice/crio-deaa01a425b47018e7cfda99134fcbece3c04f15b53a8998785ba3fa901b1434 WatchSource:0}: Error finding container deaa01a425b47018e7cfda99134fcbece3c04f15b53a8998785ba3fa901b1434: Status 404 returned error can't find the container with id deaa01a425b47018e7cfda99134fcbece3c04f15b53a8998785ba3fa901b1434
Sep 10 22:24:14.917274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.917242    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:24:14.917357 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.917307    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:24:14.917357 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:14.917331    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:24:15.421100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:15.421063    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" event={"ID":"c022a573-7bd3-4afb-bb5f-7da6e320fff7","Type":"ContainerStarted","Data":"ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab"}
Sep 10 22:24:15.421528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:15.421157    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" event={"ID":"c022a573-7bd3-4afb-bb5f-7da6e320fff7","Type":"ContainerStarted","Data":"deaa01a425b47018e7cfda99134fcbece3c04f15b53a8998785ba3fa901b1434"}
Sep 10 22:24:15.440897 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:15.440829    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podStartSLOduration=1.440811147 podStartE2EDuration="1.440811147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:24:15.440315497 +0000 UTC m=+1124.824820934" watchObservedRunningTime="2026-09-10 22:24:15.440811147 +0000 UTC m=+1124.825316572"
Sep 10 22:24:15.634616 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:15.634576    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:15.639641 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:15.639614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:16.425453 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:16.425410    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:16.426326 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:16.426303    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:19.161271 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.161239    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"]
Sep 10 22:24:19.161639 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.161596    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy"
Sep 10 22:24:19.161639 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.161608    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="56487366-5ec6-41d9-afd0-9534f65f1b3b" containerName="istio-proxy"
Sep 10 22:24:19.164594 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.164579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.166798 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.166776    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-e367c1-dockercfg-mb48l\""
Sep 10 22:24:19.166917 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.166808    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-e367c1\""
Sep 10 22:24:19.175397 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.175375    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"]
Sep 10 22:24:19.235183 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.235153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbh4h\" (UniqueName: \"kubernetes.io/projected/69e42a52-81e6-4a82-ae0a-6af1da7794b6-kube-api-access-rbh4h\") pod \"payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.235362 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.235244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e42a52-81e6-4a82-ae0a-6af1da7794b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.336394 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.336359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbh4h\" (UniqueName: \"kubernetes.io/projected/69e42a52-81e6-4a82-ae0a-6af1da7794b6-kube-api-access-rbh4h\") pod \"payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.336569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.336429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e42a52-81e6-4a82-ae0a-6af1da7794b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.337072 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.337053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e42a52-81e6-4a82-ae0a-6af1da7794b6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.344647 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.344618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbh4h\" (UniqueName: \"kubernetes.io/projected/69e42a52-81e6-4a82-ae0a-6af1da7794b6-kube-api-access-rbh4h\") pod \"payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.475465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.475366    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:19.747610 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:19.747588    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"]
Sep 10 22:24:19.749767 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:24:19.749729    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e42a52_81e6_4a82_ae0a_6af1da7794b6.slice/crio-b858280dedcdb162266231a4f7415aa974e964d7b13ed7dfc25bc26b5d219820 WatchSource:0}: Error finding container b858280dedcdb162266231a4f7415aa974e964d7b13ed7dfc25bc26b5d219820: Status 404 returned error can't find the container with id b858280dedcdb162266231a4f7415aa974e964d7b13ed7dfc25bc26b5d219820
Sep 10 22:24:20.445341 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:20.445311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs" event={"ID":"69e42a52-81e6-4a82-ae0a-6af1da7794b6","Type":"ContainerStarted","Data":"bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d"}
Sep 10 22:24:20.445902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:20.445359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs" event={"ID":"69e42a52-81e6-4a82-ae0a-6af1da7794b6","Type":"ContainerStarted","Data":"b858280dedcdb162266231a4f7415aa974e964d7b13ed7dfc25bc26b5d219820"}
Sep 10 22:24:20.445902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:20.445469    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:20.463913 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:20.463864    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs" podStartSLOduration=1.463850104 podStartE2EDuration="1.463850104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:24:20.462775511 +0000 UTC m=+1129.847280937" watchObservedRunningTime="2026-09-10 22:24:20.463850104 +0000 UTC m=+1129.848355528"
Sep 10 22:24:21.709890 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:21.709859    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:24:30.345049 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.345008    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"]
Sep 10 22:24:30.348288 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.348266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.350607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.350581    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-4a864c\""
Sep 10 22:24:30.350740 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.350635    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-4a864c-dockercfg-r522v\""
Sep 10 22:24:30.361655 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.361623    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"]
Sep 10 22:24:30.442005 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.441966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17626a83-4c8a-4678-a3e9-4c78d16126f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.442188 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.442132    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qxb\" (UniqueName: \"kubernetes.io/projected/17626a83-4c8a-4678-a3e9-4c78d16126f5-kube-api-access-j6qxb\") pod \"payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.542956 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.542928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qxb\" (UniqueName: \"kubernetes.io/projected/17626a83-4c8a-4678-a3e9-4c78d16126f5-kube-api-access-j6qxb\") pod \"payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.543168 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.542979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17626a83-4c8a-4678-a3e9-4c78d16126f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.543649 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.543624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17626a83-4c8a-4678-a3e9-4c78d16126f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.551067 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.551034    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qxb\" (UniqueName: \"kubernetes.io/projected/17626a83-4c8a-4678-a3e9-4c78d16126f5-kube-api-access-j6qxb\") pod \"payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.658796 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.658702    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:30.930176 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:30.930143    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"]
Sep 10 22:24:30.932304 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:24:30.932274    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17626a83_4c8a_4678_a3e9_4c78d16126f5.slice/crio-4d6a68ab8877e56f9de6782def86b4e90b54215854c34f01ea5820430cc5e2fd WatchSource:0}: Error finding container 4d6a68ab8877e56f9de6782def86b4e90b54215854c34f01ea5820430cc5e2fd: Status 404 returned error can't find the container with id 4d6a68ab8877e56f9de6782def86b4e90b54215854c34f01ea5820430cc5e2fd
Sep 10 22:24:31.451250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:31.451213    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:24:31.504164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:31.504129    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p" event={"ID":"17626a83-4c8a-4678-a3e9-4c78d16126f5","Type":"ContainerStarted","Data":"9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c"}
Sep 10 22:24:31.504380 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:31.504191    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p" event={"ID":"17626a83-4c8a-4678-a3e9-4c78d16126f5","Type":"ContainerStarted","Data":"4d6a68ab8877e56f9de6782def86b4e90b54215854c34f01ea5820430cc5e2fd"}
Sep 10 22:24:31.504380 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:31.504242    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:31.524856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:31.524784    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p" podStartSLOduration=1.5247696899999998 podStartE2EDuration="1.52476969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:24:31.524038379 +0000 UTC m=+1140.908543805" watchObservedRunningTime="2026-09-10 22:24:31.52476969 +0000 UTC m=+1140.909275114"
Sep 10 22:24:31.923155 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:31.922745    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:60480->10.134.0.61:15021: read: connection reset by peer" start-of-body=
Sep 10 22:24:31.923155 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:31.922818    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": read tcp 10.134.0.2:60480->10.134.0.61:15021: read: connection reset by peer"
Sep 10 22:24:32.513043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:32.513013    2563 generic.go:356] "Generic (PLEG): container finished" podID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerID="ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab" exitCode=0
Sep 10 22:24:32.513479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:32.513086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" event={"ID":"c022a573-7bd3-4afb-bb5f-7da6e320fff7","Type":"ContainerDied","Data":"ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab"}
Sep 10 22:24:32.513570 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:32.513555    2563 scope.go:117] "RemoveContainer" containerID="ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab"
Sep 10 22:24:32.514530 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:32.514505    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:24:32.514594 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:32.514566    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:24:32.514631 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:32.514595    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:24:33.411285 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:33.411256    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:24:33.523783 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:33.523736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" event={"ID":"c022a573-7bd3-4afb-bb5f-7da6e320fff7","Type":"ContainerStarted","Data":"eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381"}
Sep 10 22:24:33.634988 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:33.634945    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:33.635198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:33.635079    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 22:24:33.635198 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:33.635140    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 22:24:34.634830 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:34.634782    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 22:24:34.635359 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:34.634854    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 22:24:35.634528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:35.634484    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 22:24:35.634743 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:35.634556    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 22:24:36.634751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:36.634714    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 22:24:36.635282 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:36.634779    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 22:24:37.634203 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:37.634165    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 22:24:37.634519 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:37.634482    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 22:24:38.635938 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:38.635895    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 22:24:38.636559 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:38.636508    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 22:24:39.637338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:39.637297    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:39.637809 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:39.637657    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:39.638393 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:39.638374    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:24:42.515589 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:42.515562    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:24:47.638783 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.638742    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"]
Sep 10 22:24:47.641759 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.641736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:47.644133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.644105    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-0ada08-dockercfg-7twtz\""
Sep 10 22:24:47.644355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.644338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-0ada08\""
Sep 10 22:24:47.652686 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.652662    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"]
Sep 10 22:24:47.689342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.689310    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"]
Sep 10 22:24:47.691769 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.691751    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:47.703382 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.703354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"]
Sep 10 22:24:47.809977 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.809942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfnhl\" (UniqueName: \"kubernetes.io/projected/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-kube-api-access-dfnhl\") pod \"payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:47.810160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.809984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jrnf\" (UniqueName: \"kubernetes.io/projected/e508224a-fcce-472b-a2cd-af03f0b71d04-kube-api-access-4jrnf\") pod \"payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:47.810160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.810079    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508224a-fcce-472b-a2cd-af03f0b71d04-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:47.810160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.810112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:47.910718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.910636    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfnhl\" (UniqueName: \"kubernetes.io/projected/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-kube-api-access-dfnhl\") pod \"payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:47.910718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.910673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jrnf\" (UniqueName: \"kubernetes.io/projected/e508224a-fcce-472b-a2cd-af03f0b71d04-kube-api-access-4jrnf\") pod \"payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:47.910718 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.910712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508224a-fcce-472b-a2cd-af03f0b71d04-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:47.910962 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.910732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:47.911384 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.911366    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:47.911475 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.911426    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508224a-fcce-472b-a2cd-af03f0b71d04-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:47.922076 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.922046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jrnf\" (UniqueName: \"kubernetes.io/projected/e508224a-fcce-472b-a2cd-af03f0b71d04-kube-api-access-4jrnf\") pod \"payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:47.922251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.922233    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfnhl\" (UniqueName: \"kubernetes.io/projected/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-kube-api-access-dfnhl\") pod \"payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:47.952215 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:47.952176    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:48.002546 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.002521    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:48.224946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.224923    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"]
Sep 10 22:24:48.227203 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:24:48.227174    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode508224a_fcce_472b_a2cd_af03f0b71d04.slice/crio-fb9832cbe1f969dc867635b045ae358376eb2c5f7113d978495271153bc9d7fd WatchSource:0}: Error finding container fb9832cbe1f969dc867635b045ae358376eb2c5f7113d978495271153bc9d7fd: Status 404 returned error can't find the container with id fb9832cbe1f969dc867635b045ae358376eb2c5f7113d978495271153bc9d7fd
Sep 10 22:24:48.280578 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.280553    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"]
Sep 10 22:24:48.282782 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:24:48.282757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f0bb05_10f2_4a0c_b34a_97de6cc5e25c.slice/crio-b61edda208c70015dad0806497a4c9bfaec17bd59d3c8b69dccf3308eda93db3 WatchSource:0}: Error finding container b61edda208c70015dad0806497a4c9bfaec17bd59d3c8b69dccf3308eda93db3: Status 404 returned error can't find the container with id b61edda208c70015dad0806497a4c9bfaec17bd59d3c8b69dccf3308eda93db3
Sep 10 22:24:48.601757 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.601726    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb" event={"ID":"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c","Type":"ContainerStarted","Data":"8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c"}
Sep 10 22:24:48.601968 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.601782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb" event={"ID":"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c","Type":"ContainerStarted","Data":"b61edda208c70015dad0806497a4c9bfaec17bd59d3c8b69dccf3308eda93db3"}
Sep 10 22:24:48.601968 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.601811    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:48.603588 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.603563    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp" event={"ID":"e508224a-fcce-472b-a2cd-af03f0b71d04","Type":"ContainerStarted","Data":"d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606"}
Sep 10 22:24:48.603685 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.603617    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:48.603685 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.603641    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp" event={"ID":"e508224a-fcce-472b-a2cd-af03f0b71d04","Type":"ContainerStarted","Data":"fb9832cbe1f969dc867635b045ae358376eb2c5f7113d978495271153bc9d7fd"}
Sep 10 22:24:48.620983 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.620926    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb" podStartSLOduration=1.6209057709999999 podStartE2EDuration="1.620905771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:24:48.618945708 +0000 UTC m=+1158.003451134" watchObservedRunningTime="2026-09-10 22:24:48.620905771 +0000 UTC m=+1158.005411198"
Sep 10 22:24:48.637290 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:48.637243    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp" podStartSLOduration=1.637227512 podStartE2EDuration="1.637227512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:24:48.635508791 +0000 UTC m=+1158.020014218" watchObservedRunningTime="2026-09-10 22:24:48.637227512 +0000 UTC m=+1158.021732936"
Sep 10 22:24:51.900522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:51.900425    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:24:59.515939 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.515906    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"]
Sep 10 22:24:59.518106 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.518089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.520413 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.520387    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-32343f\""
Sep 10 22:24:59.520538 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.520387    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-32343f-dockercfg-ql5dh\""
Sep 10 22:24:59.530271 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.530247    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"]
Sep 10 22:24:59.534389 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.534365    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"]
Sep 10 22:24:59.536511 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.536496    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:24:59.555277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.555246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"]
Sep 10 22:24:59.611187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.611154    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:24:59.611390 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.611344    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:24:59.618329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.618287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7bbc96c-4571-416b-aa33-ab6883775aa4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:24:59.618539 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.618361    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g56s\" (UniqueName: \"kubernetes.io/projected/c7bbc96c-4571-416b-aa33-ab6883775aa4-kube-api-access-9g56s\") pod \"payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:24:59.618539 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.618474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmt4h\" (UniqueName: \"kubernetes.io/projected/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-kube-api-access-xmt4h\") pod \"payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.618539 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.618513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.719412 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.719376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7bbc96c-4571-416b-aa33-ab6883775aa4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:24:59.719412 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.719413    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g56s\" (UniqueName: \"kubernetes.io/projected/c7bbc96c-4571-416b-aa33-ab6883775aa4-kube-api-access-9g56s\") pod \"payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:24:59.719678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.719486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmt4h\" (UniqueName: \"kubernetes.io/projected/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-kube-api-access-xmt4h\") pod \"payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.719678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.719525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.720200 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.720174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7bbc96c-4571-416b-aa33-ab6883775aa4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:24:59.720306 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.720261    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.730162 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.730132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmt4h\" (UniqueName: \"kubernetes.io/projected/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-kube-api-access-xmt4h\") pod \"payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.730257 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.730243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g56s\" (UniqueName: \"kubernetes.io/projected/c7bbc96c-4571-416b-aa33-ab6883775aa4-kube-api-access-9g56s\") pod \"payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:24:59.828669 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.828593    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:24:59.846277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:24:59.846240    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:25:00.102470 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.102429    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"]
Sep 10 22:25:00.104503 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:00.104473    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda272dd6c_e87d_4acf_9ca5_ffa1023f7209.slice/crio-197c33fa98b4e3dbe8fe6981ccc67c17eeeea8f9f6e29bb8784be17c98f1e5f0 WatchSource:0}: Error finding container 197c33fa98b4e3dbe8fe6981ccc67c17eeeea8f9f6e29bb8784be17c98f1e5f0: Status 404 returned error can't find the container with id 197c33fa98b4e3dbe8fe6981ccc67c17eeeea8f9f6e29bb8784be17c98f1e5f0
Sep 10 22:25:00.127666 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.127641    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"]
Sep 10 22:25:00.142693 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:00.142658    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7bbc96c_4571_416b_aa33_ab6883775aa4.slice/crio-621ea7494462b8601409fa755d82bbe65b1ec3479eb5445cd855034e6dacef29 WatchSource:0}: Error finding container 621ea7494462b8601409fa755d82bbe65b1ec3479eb5445cd855034e6dacef29: Status 404 returned error can't find the container with id 621ea7494462b8601409fa755d82bbe65b1ec3479eb5445cd855034e6dacef29
Sep 10 22:25:00.665615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.665575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8" event={"ID":"a272dd6c-e87d-4acf-9ca5-ffa1023f7209","Type":"ContainerStarted","Data":"8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138"}
Sep 10 22:25:00.666075 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.665650    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:25:00.666075 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.665673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8" event={"ID":"a272dd6c-e87d-4acf-9ca5-ffa1023f7209","Type":"ContainerStarted","Data":"197c33fa98b4e3dbe8fe6981ccc67c17eeeea8f9f6e29bb8784be17c98f1e5f0"}
Sep 10 22:25:00.667166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.667143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l" event={"ID":"c7bbc96c-4571-416b-aa33-ab6883775aa4","Type":"ContainerStarted","Data":"eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7"}
Sep 10 22:25:00.667304 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.667187    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l" event={"ID":"c7bbc96c-4571-416b-aa33-ab6883775aa4","Type":"ContainerStarted","Data":"621ea7494462b8601409fa755d82bbe65b1ec3479eb5445cd855034e6dacef29"}
Sep 10 22:25:00.667304 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.667221    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:25:00.684368 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.684323    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8" podStartSLOduration=1.6843102170000002 podStartE2EDuration="1.684310217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:00.682724554 +0000 UTC m=+1170.067229978" watchObservedRunningTime="2026-09-10 22:25:00.684310217 +0000 UTC m=+1170.068815642"
Sep 10 22:25:00.698962 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:00.698922    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l" podStartSLOduration=1.698907889 podStartE2EDuration="1.698907889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:24:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:00.697656152 +0000 UTC m=+1170.082161579" watchObservedRunningTime="2026-09-10 22:25:00.698907889 +0000 UTC m=+1170.083413314"
Sep 10 22:25:03.702823 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:03.702792    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:25:11.674362 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:11.674331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:25:11.674763 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:11.674409    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:25:17.099139 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.099109    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"]
Sep 10 22:25:17.102264 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.102245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.104592 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.104551    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"kube-root-ca.crt\""
Sep 10 22:25:17.104711 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.104556    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"default-dockercfg-rxtzj\""
Sep 10 22:25:17.105301 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.105281    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"openshift-service-ca.crt\""
Sep 10 22:25:17.105366 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.105283    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"auth-test-model-e367c1-kserve-self-signed-certs\""
Sep 10 22:25:17.114117 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.114089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"]
Sep 10 22:25:17.174788 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.174755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz2vv\" (UniqueName: \"kubernetes.io/projected/6388d6d1-7612-4b8d-b459-7528fcec4c26-kube-api-access-bz2vv\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.174959 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.174801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6388d6d1-7612-4b8d-b459-7528fcec4c26-tls-certs\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.174959 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.174945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-tmp-dir\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.175079 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.174976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-model-cache\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.175140 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.175085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-dshm\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.175140 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.175108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-home\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz2vv\" (UniqueName: \"kubernetes.io/projected/6388d6d1-7612-4b8d-b459-7528fcec4c26-kube-api-access-bz2vv\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276379 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6388d6d1-7612-4b8d-b459-7528fcec4c26-tls-certs\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276379 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-tmp-dir\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-model-cache\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276584 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-dshm\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276642 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-home\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276700 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-tmp-dir\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-model-cache\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.276924 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.276903    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-home\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.278755 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.278734    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-dshm\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.278850 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.278831    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6388d6d1-7612-4b8d-b459-7528fcec4c26-tls-certs\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.283426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.283404    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz2vv\" (UniqueName: \"kubernetes.io/projected/6388d6d1-7612-4b8d-b459-7528fcec4c26-kube-api-access-bz2vv\") pod \"auth-test-model-e367c1-kserve-579569b675-hcglw\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.413671 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.413592    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:17.689401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.689374    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"]
Sep 10 22:25:17.691412 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:17.691387    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6388d6d1_7612_4b8d_b459_7528fcec4c26.slice/crio-ff5609a514223a98c64ee31b65a7a0975cbd3a780712d69be0fb43181ec02ca8 WatchSource:0}: Error finding container ff5609a514223a98c64ee31b65a7a0975cbd3a780712d69be0fb43181ec02ca8: Status 404 returned error can't find the container with id ff5609a514223a98c64ee31b65a7a0975cbd3a780712d69be0fb43181ec02ca8
Sep 10 22:25:17.756302 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.756266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw" event={"ID":"6388d6d1-7612-4b8d-b459-7528fcec4c26","Type":"ContainerStarted","Data":"e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628"}
Sep 10 22:25:17.756479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.756317    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw" event={"ID":"6388d6d1-7612-4b8d-b459-7528fcec4c26","Type":"ContainerStarted","Data":"ff5609a514223a98c64ee31b65a7a0975cbd3a780712d69be0fb43181ec02ca8"}
Sep 10 22:25:17.774093 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:17.774043    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw" podStartSLOduration=0.774011937 podStartE2EDuration="774.011937ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:17.771852419 +0000 UTC m=+1187.156357845" watchObservedRunningTime="2026-09-10 22:25:17.774011937 +0000 UTC m=+1187.158517362"
Sep 10 22:25:20.909632 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:20.909596    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:25:23.544153 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.544113    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-abb18a"]
Sep 10 22:25:23.547313 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.547295    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:23.549682 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.549655    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qckd6\""
Sep 10 22:25:23.551377 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.551351    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-abb18a"]
Sep 10 22:25:23.644398 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.644357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h25k\" (UniqueName: \"kubernetes.io/projected/d0d71b6e-ea19-47e3-aef0-0ec8c69381ed-kube-api-access-6h25k\") pod \"test-curl-7605-abb18a\" (UID: \"d0d71b6e-ea19-47e3-aef0-0ec8c69381ed\") " pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:23.745083 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.745059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h25k\" (UniqueName: \"kubernetes.io/projected/d0d71b6e-ea19-47e3-aef0-0ec8c69381ed-kube-api-access-6h25k\") pod \"test-curl-7605-abb18a\" (UID: \"d0d71b6e-ea19-47e3-aef0-0ec8c69381ed\") " pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:23.753097 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.753065    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h25k\" (UniqueName: \"kubernetes.io/projected/d0d71b6e-ea19-47e3-aef0-0ec8c69381ed-kube-api-access-6h25k\") pod \"test-curl-7605-abb18a\" (UID: \"d0d71b6e-ea19-47e3-aef0-0ec8c69381ed\") " pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:23.858520 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:23.858412    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:24.124871 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:24.124848    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-abb18a"]
Sep 10 22:25:24.126977 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:24.126945    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0d71b6e_ea19_47e3_aef0_0ec8c69381ed.slice/crio-68ab6cef75a49a5973272bb0e30695b96e8f7b79a096bcd6fed4b9ed4c4661e9 WatchSource:0}: Error finding container 68ab6cef75a49a5973272bb0e30695b96e8f7b79a096bcd6fed4b9ed4c4661e9: Status 404 returned error can't find the container with id 68ab6cef75a49a5973272bb0e30695b96e8f7b79a096bcd6fed4b9ed4c4661e9
Sep 10 22:25:24.795984 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:24.795943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-abb18a" event={"ID":"d0d71b6e-ea19-47e3-aef0-0ec8c69381ed","Type":"ContainerStarted","Data":"6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b"}
Sep 10 22:25:24.795984 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:24.795986    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-abb18a" event={"ID":"d0d71b6e-ea19-47e3-aef0-0ec8c69381ed","Type":"ContainerStarted","Data":"68ab6cef75a49a5973272bb0e30695b96e8f7b79a096bcd6fed4b9ed4c4661e9"}
Sep 10 22:25:24.810428 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:24.810386    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7605-abb18a" podStartSLOduration=1.810372503 podStartE2EDuration="1.810372503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:24.808102531 +0000 UTC m=+1194.192607955" watchObservedRunningTime="2026-09-10 22:25:24.810372503 +0000 UTC m=+1194.194877927"
Sep 10 22:25:25.399561 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.399522    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-abb18a"]
Sep 10 22:25:25.401637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.401611    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-abb18a"]
Sep 10 22:25:25.515174 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.515147    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-7da56f"]
Sep 10 22:25:25.515526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.515512    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" containerName="test-curl-7605-abb18a"
Sep 10 22:25:25.515576 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.515527    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" containerName="test-curl-7605-abb18a"
Sep 10 22:25:25.515613 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.515604    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" containerName="test-curl-7605-abb18a"
Sep 10 22:25:25.517399 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.517384    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:25.523910 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.523883    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-7da56f"]
Sep 10 22:25:25.662207 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.662173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2476w\" (UniqueName: \"kubernetes.io/projected/58704377-9007-4765-9be3-f4700ec3f81c-kube-api-access-2476w\") pod \"test-curl-7605-7da56f\" (UID: \"58704377-9007-4765-9be3-f4700ec3f81c\") " pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:25.762786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.762756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2476w\" (UniqueName: \"kubernetes.io/projected/58704377-9007-4765-9be3-f4700ec3f81c-kube-api-access-2476w\") pod \"test-curl-7605-7da56f\" (UID: \"58704377-9007-4765-9be3-f4700ec3f81c\") " pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:25.770989 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.770929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2476w\" (UniqueName: \"kubernetes.io/projected/58704377-9007-4765-9be3-f4700ec3f81c-kube-api-access-2476w\") pod \"test-curl-7605-7da56f\" (UID: \"58704377-9007-4765-9be3-f4700ec3f81c\") " pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:25.801031 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:25.801002    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7605-abb18a" container="test-curl-7605-abb18a"
Sep 10 22:25:25.827133 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:25.827115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:26.094221 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:26.094195    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-7da56f"]
Sep 10 22:25:26.096430 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:26.096404    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58704377_9007_4765_9be3_f4700ec3f81c.slice/crio-9edd693108505dd01d176c98430864fdf6ca4af26fb6a1692bba2c147cae6d7f WatchSource:0}: Error finding container 9edd693108505dd01d176c98430864fdf6ca4af26fb6a1692bba2c147cae6d7f: Status 404 returned error can't find the container with id 9edd693108505dd01d176c98430864fdf6ca4af26fb6a1692bba2c147cae6d7f
Sep 10 22:25:26.807392 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:26.807359    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-7da56f" event={"ID":"58704377-9007-4765-9be3-f4700ec3f81c","Type":"ContainerStarted","Data":"aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579"}
Sep 10 22:25:26.807392 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:26.807398    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-7da56f" event={"ID":"58704377-9007-4765-9be3-f4700ec3f81c","Type":"ContainerStarted","Data":"9edd693108505dd01d176c98430864fdf6ca4af26fb6a1692bba2c147cae6d7f"}
Sep 10 22:25:26.807902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:26.807501    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7605-abb18a" podUID="d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" containerName="test-curl-7605-abb18a" containerID="cri-o://6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b" gracePeriod=2
Sep 10 22:25:26.820860 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:26.820810    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7605-7da56f" podStartSLOduration=1.820790722 podStartE2EDuration="1.820790722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:26.819210267 +0000 UTC m=+1196.203715695" watchObservedRunningTime="2026-09-10 22:25:26.820790722 +0000 UTC m=+1196.205296152"
Sep 10 22:25:26.820998 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:26.820839    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-abb18a\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:27.406805 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.406771    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-7da56f"]
Sep 10 22:25:27.408850 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.408802    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-7da56f"]
Sep 10 22:25:27.414144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.414112    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:27.414286 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.414169    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:27.432167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.432145    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:27.524454 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.524406    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-5973ee"]
Sep 10 22:25:27.524872 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.524854    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58704377-9007-4765-9be3-f4700ec3f81c" containerName="test-curl-7605-7da56f"
Sep 10 22:25:27.524964 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.524875    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="58704377-9007-4765-9be3-f4700ec3f81c" containerName="test-curl-7605-7da56f"
Sep 10 22:25:27.525018 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.524986    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="58704377-9007-4765-9be3-f4700ec3f81c" containerName="test-curl-7605-7da56f"
Sep 10 22:25:27.526789 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.526771    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:27.531326 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.531301    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-5973ee"]
Sep 10 22:25:27.681127 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.681095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc9zj\" (UniqueName: \"kubernetes.io/projected/eb19947b-bb49-421d-bf95-0c0d77f8487b-kube-api-access-rc9zj\") pod \"test-curl-7605-5973ee\" (UID: \"eb19947b-bb49-421d-bf95-0c0d77f8487b\") " pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:27.782702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.782630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc9zj\" (UniqueName: \"kubernetes.io/projected/eb19947b-bb49-421d-bf95-0c0d77f8487b-kube-api-access-rc9zj\") pod \"test-curl-7605-5973ee\" (UID: \"eb19947b-bb49-421d-bf95-0c0d77f8487b\") " pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:27.790238 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.790212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc9zj\" (UniqueName: \"kubernetes.io/projected/eb19947b-bb49-421d-bf95-0c0d77f8487b-kube-api-access-rc9zj\") pod \"test-curl-7605-5973ee\" (UID: \"eb19947b-bb49-421d-bf95-0c0d77f8487b\") " pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:27.812044 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:27.812016    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7605-7da56f" container="test-curl-7605-7da56f"
Sep 10 22:25:27.823098 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.823075    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:25:27.837351 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:27.837322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:28.135912 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:28.135888    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-5973ee"]
Sep 10 22:25:28.138029 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:28.138000    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb19947b_bb49_421d_bf95_0c0d77f8487b.slice/crio-313b1857aa0ac515ef13b7d7bb9e7be698eb108495dbc9bfb24d8fd072c294c6 WatchSource:0}: Error finding container 313b1857aa0ac515ef13b7d7bb9e7be698eb108495dbc9bfb24d8fd072c294c6: Status 404 returned error can't find the container with id 313b1857aa0ac515ef13b7d7bb9e7be698eb108495dbc9bfb24d8fd072c294c6
Sep 10 22:25:28.819152 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:28.819121    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-5973ee" event={"ID":"eb19947b-bb49-421d-bf95-0c0d77f8487b","Type":"ContainerStarted","Data":"aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729"}
Sep 10 22:25:28.819627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:28.819165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-5973ee" event={"ID":"eb19947b-bb49-421d-bf95-0c0d77f8487b","Type":"ContainerStarted","Data":"313b1857aa0ac515ef13b7d7bb9e7be698eb108495dbc9bfb24d8fd072c294c6"}
Sep 10 22:25:28.819627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:28.819486    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7605-7da56f" podUID="58704377-9007-4765-9be3-f4700ec3f81c" containerName="test-curl-7605-7da56f" containerID="cri-o://aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579" gracePeriod=2
Sep 10 22:25:28.833850 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:28.833814    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-7da56f\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="58704377-9007-4765-9be3-f4700ec3f81c" pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:28.833996 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:28.833965    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7605-5973ee" podStartSLOduration=1.833953799 podStartE2EDuration="1.833953799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:28.832066683 +0000 UTC m=+1198.216572108" watchObservedRunningTime="2026-09-10 22:25:28.833953799 +0000 UTC m=+1198.218459260"
Sep 10 22:25:28.976028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:28.976004    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:28.977857 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:28.977832    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-7da56f\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="58704377-9007-4765-9be3-f4700ec3f81c" pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:29.094570 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.094497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h25k\" (UniqueName: \"kubernetes.io/projected/d0d71b6e-ea19-47e3-aef0-0ec8c69381ed-kube-api-access-6h25k\") pod \"d0d71b6e-ea19-47e3-aef0-0ec8c69381ed\" (UID: \"d0d71b6e-ea19-47e3-aef0-0ec8c69381ed\") "
Sep 10 22:25:29.096550 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.096517    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d71b6e-ea19-47e3-aef0-0ec8c69381ed-kube-api-access-6h25k" (OuterVolumeSpecName: "kube-api-access-6h25k") pod "d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" (UID: "d0d71b6e-ea19-47e3-aef0-0ec8c69381ed"). InnerVolumeSpecName "kube-api-access-6h25k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:25:29.110994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.110967    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" path="/var/lib/kubelet/pods/d0d71b6e-ea19-47e3-aef0-0ec8c69381ed/volumes"
Sep 10 22:25:29.196065 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.196041    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h25k\" (UniqueName: \"kubernetes.io/projected/d0d71b6e-ea19-47e3-aef0-0ec8c69381ed-kube-api-access-6h25k\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:25:29.397993 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.397912    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-5973ee"]
Sep 10 22:25:29.401793 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.401750    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-5973ee"]
Sep 10 22:25:29.518986 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.518949    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7605-0c1071"]
Sep 10 22:25:29.519416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.519398    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" containerName="test-curl-7605-5973ee"
Sep 10 22:25:29.519543 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.519418    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" containerName="test-curl-7605-5973ee"
Sep 10 22:25:29.519543 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.519527    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" containerName="test-curl-7605-5973ee"
Sep 10 22:25:29.521845 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.521824    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:29.527565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.527541    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-0c1071"]
Sep 10 22:25:29.699992 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.699907    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w8pp\" (UniqueName: \"kubernetes.io/projected/9848d084-0114-48ab-a4b3-b4c32dc98b2a-kube-api-access-9w8pp\") pod \"test-curl-7605-0c1071\" (UID: \"9848d084-0114-48ab-a4b3-b4c32dc98b2a\") " pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:29.801113 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.801079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w8pp\" (UniqueName: \"kubernetes.io/projected/9848d084-0114-48ab-a4b3-b4c32dc98b2a-kube-api-access-9w8pp\") pod \"test-curl-7605-0c1071\" (UID: \"9848d084-0114-48ab-a4b3-b4c32dc98b2a\") " pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:29.808753 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.808720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w8pp\" (UniqueName: \"kubernetes.io/projected/9848d084-0114-48ab-a4b3-b4c32dc98b2a-kube-api-access-9w8pp\") pod \"test-curl-7605-0c1071\" (UID: \"9848d084-0114-48ab-a4b3-b4c32dc98b2a\") " pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:29.825042 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.825017    2563 generic.go:356] "Generic (PLEG): container finished" podID="d0d71b6e-ea19-47e3-aef0-0ec8c69381ed" containerID="6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b" exitCode=137
Sep 10 22:25:29.825429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.825072    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-abb18a"
Sep 10 22:25:29.825429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.825114    2563 scope.go:117] "RemoveContainer" containerID="6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b"
Sep 10 22:25:29.825429 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:29.825394    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7605-5973ee" container="test-curl-7605-5973ee"
Sep 10 22:25:29.832004 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.831981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:29.833669 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.833562    2563 scope.go:117] "RemoveContainer" containerID="6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b"
Sep 10 22:25:29.833869 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:29.833848    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b\": container with ID starting with 6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b not found: ID does not exist" containerID="6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b"
Sep 10 22:25:29.833949 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:29.833878    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b"} err="failed to get container status \"6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b\": rpc error: code = NotFound desc = could not find container \"6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b\": container with ID starting with 6b37a0e1b12786d432f95071a3c9d4fd8eda33b03bddca1ef1d5e4ddc590f74b not found: ID does not exist"
Sep 10 22:25:30.100645 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:30.100622    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7605-0c1071"]
Sep 10 22:25:30.103093 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:30.103067    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9848d084_0114_48ab_a4b3_b4c32dc98b2a.slice/crio-53def557c7f3920e2344091d99a030c58ccb879d4d28311e337b3b267334d1a9 WatchSource:0}: Error finding container 53def557c7f3920e2344091d99a030c58ccb879d4d28311e337b3b267334d1a9: Status 404 returned error can't find the container with id 53def557c7f3920e2344091d99a030c58ccb879d4d28311e337b3b267334d1a9
Sep 10 22:25:30.832201 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:30.832173    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-0c1071" event={"ID":"9848d084-0114-48ab-a4b3-b4c32dc98b2a","Type":"ContainerStarted","Data":"56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3"}
Sep 10 22:25:30.832590 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:30.832215    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7605-0c1071" event={"ID":"9848d084-0114-48ab-a4b3-b4c32dc98b2a","Type":"ContainerStarted","Data":"53def557c7f3920e2344091d99a030c58ccb879d4d28311e337b3b267334d1a9"}
Sep 10 22:25:30.833317 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:30.833291    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7605-5973ee" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" containerName="test-curl-7605-5973ee" containerID="cri-o://aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729" gracePeriod=2
Sep 10 22:25:30.844806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:30.844758    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7605-0c1071" podStartSLOduration=1.84474011 podStartE2EDuration="1.84474011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:30.844065075 +0000 UTC m=+1200.228570512" watchObservedRunningTime="2026-09-10 22:25:30.84474011 +0000 UTC m=+1200.229245536"
Sep 10 22:25:30.845899 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:30.845866    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-5973ee\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:31.001547 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.001524    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:31.005298 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:31.005274    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-5973ee\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:31.014149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.014120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2476w\" (UniqueName: \"kubernetes.io/projected/58704377-9007-4765-9be3-f4700ec3f81c-kube-api-access-2476w\") pod \"58704377-9007-4765-9be3-f4700ec3f81c\" (UID: \"58704377-9007-4765-9be3-f4700ec3f81c\") "
Sep 10 22:25:31.016181 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.016153    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58704377-9007-4765-9be3-f4700ec3f81c-kube-api-access-2476w" (OuterVolumeSpecName: "kube-api-access-2476w") pod "58704377-9007-4765-9be3-f4700ec3f81c" (UID: "58704377-9007-4765-9be3-f4700ec3f81c"). InnerVolumeSpecName "kube-api-access-2476w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:25:31.108664 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:31.108557    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-5973ee\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:31.111368 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.111342    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58704377-9007-4765-9be3-f4700ec3f81c" path="/var/lib/kubelet/pods/58704377-9007-4765-9be3-f4700ec3f81c/volumes"
Sep 10 22:25:31.114860 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.114836    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2476w\" (UniqueName: \"kubernetes.io/projected/58704377-9007-4765-9be3-f4700ec3f81c-kube-api-access-2476w\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:25:31.406873 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.406798    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7605-0c1071"]
Sep 10 22:25:31.415253 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.415215    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7605-0c1071"]
Sep 10 22:25:31.839229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.839193    2563 generic.go:356] "Generic (PLEG): container finished" podID="58704377-9007-4765-9be3-f4700ec3f81c" containerID="aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579" exitCode=137
Sep 10 22:25:31.839716 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.839242    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-7da56f"
Sep 10 22:25:31.839716 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.839294    2563 scope.go:117] "RemoveContainer" containerID="aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579"
Sep 10 22:25:31.839716 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:31.839549    2563 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7605-0c1071" container="test-curl-7605-0c1071"
Sep 10 22:25:31.847466 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.847429    2563 scope.go:117] "RemoveContainer" containerID="aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579"
Sep 10 22:25:31.847751 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:31.847733    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579\": container with ID starting with aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579 not found: ID does not exist" containerID="aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579"
Sep 10 22:25:31.847814 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:31.847764    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579"} err="failed to get container status \"aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579\": rpc error: code = NotFound desc = could not find container \"aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579\": container with ID starting with aca235ef2245e0522e56fea7324dec26b30ae6931c301d6e675b5f4018ca0579 not found: ID does not exist"
Sep 10 22:25:32.845138 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:32.845111    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7605-0c1071" podUID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" containerName="test-curl-7605-0c1071" containerID="cri-o://56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3" gracePeriod=2
Sep 10 22:25:32.846926 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:32.846887    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-0c1071\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:33.005741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.005716    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:33.007903 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:33.007873    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7605-0c1071\" is forbidden: User \"system:node:ip-10-0-137-25.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-25.ec2.internal' and this object" podUID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:33.031257 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.031236    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc9zj\" (UniqueName: \"kubernetes.io/projected/eb19947b-bb49-421d-bf95-0c0d77f8487b-kube-api-access-rc9zj\") pod \"eb19947b-bb49-421d-bf95-0c0d77f8487b\" (UID: \"eb19947b-bb49-421d-bf95-0c0d77f8487b\") "
Sep 10 22:25:33.033392 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.033365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb19947b-bb49-421d-bf95-0c0d77f8487b-kube-api-access-rc9zj" (OuterVolumeSpecName: "kube-api-access-rc9zj") pod "eb19947b-bb49-421d-bf95-0c0d77f8487b" (UID: "eb19947b-bb49-421d-bf95-0c0d77f8487b"). InnerVolumeSpecName "kube-api-access-rc9zj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:25:33.110991 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.110928    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb19947b-bb49-421d-bf95-0c0d77f8487b" path="/var/lib/kubelet/pods/eb19947b-bb49-421d-bf95-0c0d77f8487b/volumes"
Sep 10 22:25:33.132213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.132191    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc9zj\" (UniqueName: \"kubernetes.io/projected/eb19947b-bb49-421d-bf95-0c0d77f8487b-kube-api-access-rc9zj\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:25:33.854354 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.854322    2563 generic.go:356] "Generic (PLEG): container finished" podID="eb19947b-bb49-421d-bf95-0c0d77f8487b" containerID="aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729" exitCode=137
Sep 10 22:25:33.854828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.854364    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-5973ee"
Sep 10 22:25:33.854828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.854423    2563 scope.go:117] "RemoveContainer" containerID="aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729"
Sep 10 22:25:33.862899 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.862882    2563 scope.go:117] "RemoveContainer" containerID="aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729"
Sep 10 22:25:33.863175 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:33.863157    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729\": container with ID starting with aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729 not found: ID does not exist" containerID="aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729"
Sep 10 22:25:33.863218 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:33.863184    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729"} err="failed to get container status \"aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729\": rpc error: code = NotFound desc = could not find container \"aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729\": container with ID starting with aad5deddc441c7f3bffc94c2260e44068c84cd973bc261ab9379dd313a56c729 not found: ID does not exist"
Sep 10 22:25:35.018043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.018010    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:35.046716 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.046692    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9w8pp\" (UniqueName: \"kubernetes.io/projected/9848d084-0114-48ab-a4b3-b4c32dc98b2a-kube-api-access-9w8pp\") pod \"9848d084-0114-48ab-a4b3-b4c32dc98b2a\" (UID: \"9848d084-0114-48ab-a4b3-b4c32dc98b2a\") "
Sep 10 22:25:35.058607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.049102    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9848d084-0114-48ab-a4b3-b4c32dc98b2a-kube-api-access-9w8pp" (OuterVolumeSpecName: "kube-api-access-9w8pp") pod "9848d084-0114-48ab-a4b3-b4c32dc98b2a" (UID: "9848d084-0114-48ab-a4b3-b4c32dc98b2a"). InnerVolumeSpecName "kube-api-access-9w8pp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:25:35.110384 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.110304    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" path="/var/lib/kubelet/pods/9848d084-0114-48ab-a4b3-b4c32dc98b2a/volumes"
Sep 10 22:25:35.147343 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.147318    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w8pp\" (UniqueName: \"kubernetes.io/projected/9848d084-0114-48ab-a4b3-b4c32dc98b2a-kube-api-access-9w8pp\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:25:35.867410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.867371    2563 generic.go:356] "Generic (PLEG): container finished" podID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" containerID="56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3" exitCode=137
Sep 10 22:25:35.867630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.867416    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7605-0c1071"
Sep 10 22:25:35.867630 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.867496    2563 scope.go:117] "RemoveContainer" containerID="56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3"
Sep 10 22:25:35.876054 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.876034    2563 scope.go:117] "RemoveContainer" containerID="56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3"
Sep 10 22:25:35.876371 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:25:35.876345    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3\": container with ID starting with 56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3 not found: ID does not exist" containerID="56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3"
Sep 10 22:25:35.876418 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:35.876382    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3"} err="failed to get container status \"56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3\": rpc error: code = NotFound desc = could not find container \"56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3\": container with ID starting with 56c22d147adbad0f77de2d79dec4295f22fabb949d0fb13473202f5dc0130dd3 not found: ID does not exist"
Sep 10 22:25:36.934218 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.934183    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"]
Sep 10 22:25:36.934608 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.934584    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" containerName="test-curl-7605-0c1071"
Sep 10 22:25:36.934608 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.934595    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" containerName="test-curl-7605-0c1071"
Sep 10 22:25:36.934688 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.934671    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9848d084-0114-48ab-a4b3-b4c32dc98b2a" containerName="test-curl-7605-0c1071"
Sep 10 22:25:36.938962 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.938943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:36.943044 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.943016    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-853c9ae4\""
Sep 10 22:25:36.957233 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.957202    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"]
Sep 10 22:25:36.962114 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.962088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4618db54-bb56-4607-8a88-6617d2abf204-plugins-config-volume\") pod \"payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") " pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:36.962268 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:36.962201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs4xh\" (UniqueName: \"kubernetes.io/projected/4618db54-bb56-4607-8a88-6617d2abf204-kube-api-access-xs4xh\") pod \"payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") " pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:37.062940 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.062893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4618db54-bb56-4607-8a88-6617d2abf204-plugins-config-volume\") pod \"payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") " pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:37.063132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.063008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs4xh\" (UniqueName: \"kubernetes.io/projected/4618db54-bb56-4607-8a88-6617d2abf204-kube-api-access-xs4xh\") pod \"payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") " pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:37.063662 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.063638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4618db54-bb56-4607-8a88-6617d2abf204-plugins-config-volume\") pod \"payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") " pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:37.071233 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.071204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs4xh\" (UniqueName: \"kubernetes.io/projected/4618db54-bb56-4607-8a88-6617d2abf204-kube-api-access-xs4xh\") pod \"payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") " pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:37.250024 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.249987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:37.521804 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.521731    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"]
Sep 10 22:25:37.524363 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:37.524332    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4618db54_bb56_4607_8a88_6617d2abf204.slice/crio-458adc4b31d65eea85b680a3693641ce41ab1ff9968dcfbb0c66e7b8e46f97d5 WatchSource:0}: Error finding container 458adc4b31d65eea85b680a3693641ce41ab1ff9968dcfbb0c66e7b8e46f97d5: Status 404 returned error can't find the container with id 458adc4b31d65eea85b680a3693641ce41ab1ff9968dcfbb0c66e7b8e46f97d5
Sep 10 22:25:37.880202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.880104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq" event={"ID":"4618db54-bb56-4607-8a88-6617d2abf204","Type":"ContainerStarted","Data":"6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63"}
Sep 10 22:25:37.880202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.880161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq" event={"ID":"4618db54-bb56-4607-8a88-6617d2abf204","Type":"ContainerStarted","Data":"458adc4b31d65eea85b680a3693641ce41ab1ff9968dcfbb0c66e7b8e46f97d5"}
Sep 10 22:25:37.880426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.880257    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:37.896795 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:37.896743    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq" podStartSLOduration=1.896728086 podStartE2EDuration="1.896728086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:37.895005937 +0000 UTC m=+1207.279511363" watchObservedRunningTime="2026-09-10 22:25:37.896728086 +0000 UTC m=+1207.281233512"
Sep 10 22:25:41.805419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:41.805382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:25:47.004864 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.004835    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"]
Sep 10 22:25:47.008560 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.008543    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.011852 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.011827    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"auth-test-model-0ada08-kserve-self-signed-certs\""
Sep 10 22:25:47.011991 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.011854    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"kube-root-ca.crt\""
Sep 10 22:25:47.011991 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.011828    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"openshift-service-ca.crt\""
Sep 10 22:25:47.011991 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.011829    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"default-dockercfg-567d7\""
Sep 10 22:25:47.020844 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.020819    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"]
Sep 10 22:25:47.048683 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.048655    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-model-cache\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.048798 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.048722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-home\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.048798 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.048756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-tmp-dir\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.048798 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.048776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxfb\" (UniqueName: \"kubernetes.io/projected/9434136e-f753-480e-bb0b-15f823ceb6a7-kube-api-access-gxxfb\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.048920 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.048827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9434136e-f753-480e-bb0b-15f823ceb6a7-tls-certs\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.048920 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.048856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-dshm\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150118 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-model-cache\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150267 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-home\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150267 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-tmp-dir\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150267 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxfb\" (UniqueName: \"kubernetes.io/projected/9434136e-f753-480e-bb0b-15f823ceb6a7-kube-api-access-gxxfb\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150267 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9434136e-f753-480e-bb0b-15f823ceb6a7-tls-certs\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150267 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-dshm\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150584 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150507    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-model-cache\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150584 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150560    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-home\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.150693 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.150612    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-tmp-dir\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.152691 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.152669    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-dshm\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.152884 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.152867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9434136e-f753-480e-bb0b-15f823ceb6a7-tls-certs\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.158792 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.158770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxfb\" (UniqueName: \"kubernetes.io/projected/9434136e-f753-480e-bb0b-15f823ceb6a7-kube-api-access-gxxfb\") pod \"auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.318100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.318009    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:47.595640 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.595615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"]
Sep 10 22:25:47.597899 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:25:47.597870    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9434136e_f753_480e_bb0b_15f823ceb6a7.slice/crio-09421e792ced9f741865be48ab829d962ba4a38bae5b823db005666408f07b8a WatchSource:0}: Error finding container 09421e792ced9f741865be48ab829d962ba4a38bae5b823db005666408f07b8a: Status 404 returned error can't find the container with id 09421e792ced9f741865be48ab829d962ba4a38bae5b823db005666408f07b8a
Sep 10 22:25:47.934673 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.934594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5" event={"ID":"9434136e-f753-480e-bb0b-15f823ceb6a7","Type":"ContainerStarted","Data":"3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681"}
Sep 10 22:25:47.934673 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.934634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5" event={"ID":"9434136e-f753-480e-bb0b-15f823ceb6a7","Type":"ContainerStarted","Data":"09421e792ced9f741865be48ab829d962ba4a38bae5b823db005666408f07b8a"}
Sep 10 22:25:47.951741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:47.951692    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5" podStartSLOduration=1.951675174 podStartE2EDuration="1.951675174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:25:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:25:47.951111258 +0000 UTC m=+1217.335616684" watchObservedRunningTime="2026-09-10 22:25:47.951675174 +0000 UTC m=+1217.336180633"
Sep 10 22:25:48.888002 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:48.887970    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:25:54.011331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:54.011292    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:25:57.318465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:57.318392    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:57.318465 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:57.318478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:57.331237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:57.331211    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:25:57.998156 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:25:57.998119    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:26:04.700202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.700168    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"]
Sep 10 22:26:04.704075 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.704050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.706236 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.706208    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-853c9ae4\"/\"openshift-service-ca.crt\""
Sep 10 22:26:04.706363 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.706264    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-853c9ae4\"/\"kube-root-ca.crt\""
Sep 10 22:26:04.706948 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.706932    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-853c9ae4\"/\"e2e-lifecycle-model-853c9ae4-kserve-self-signed-certs\""
Sep 10 22:26:04.707040 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.706937    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-853c9ae4\"/\"default-dockercfg-5xdlz\""
Sep 10 22:26:04.715195 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.715171    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"]
Sep 10 22:26:04.819850 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.819819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-tmp-dir\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.820013 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.819859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m96lv\" (UniqueName: \"kubernetes.io/projected/71048100-fdb7-4e6f-b708-958faa9ec6d0-kube-api-access-m96lv\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.820013 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.819878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-model-cache\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.820144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.820008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-home\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.820144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.820047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71048100-fdb7-4e6f-b708-958faa9ec6d0-tls-certs\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.820237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.820142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-dshm\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.920802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.920771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-home\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.920818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71048100-fdb7-4e6f-b708-958faa9ec6d0-tls-certs\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.920849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-dshm\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.920865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-tmp-dir\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.920885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m96lv\" (UniqueName: \"kubernetes.io/projected/71048100-fdb7-4e6f-b708-958faa9ec6d0-kube-api-access-m96lv\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.920901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-model-cache\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.921324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-home\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921411 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.921340    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-model-cache\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.921411 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.921375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-tmp-dir\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.923432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.923409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-dshm\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.923674 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.923656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71048100-fdb7-4e6f-b708-958faa9ec6d0-tls-certs\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:04.928259 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:04.928239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m96lv\" (UniqueName: \"kubernetes.io/projected/71048100-fdb7-4e6f-b708-958faa9ec6d0-kube-api-access-m96lv\") pod \"e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") " pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:05.015643 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:05.015597    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:05.286858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:05.286830    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"]
Sep 10 22:26:05.288360 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:26:05.288333    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71048100_fdb7_4e6f_b708_958faa9ec6d0.slice/crio-e682e0a6c7eb836c705b00bc9381bfec50264a03405383f2310f430067fdb9b6 WatchSource:0}: Error finding container e682e0a6c7eb836c705b00bc9381bfec50264a03405383f2310f430067fdb9b6: Status 404 returned error can't find the container with id e682e0a6c7eb836c705b00bc9381bfec50264a03405383f2310f430067fdb9b6
Sep 10 22:26:06.030916 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:06.030880    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg" event={"ID":"71048100-fdb7-4e6f-b708-958faa9ec6d0","Type":"ContainerStarted","Data":"53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32"}
Sep 10 22:26:06.031393 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:06.030927    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg" event={"ID":"71048100-fdb7-4e6f-b708-958faa9ec6d0","Type":"ContainerStarted","Data":"e682e0a6c7eb836c705b00bc9381bfec50264a03405383f2310f430067fdb9b6"}
Sep 10 22:26:06.046212 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:06.046162    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg" podStartSLOduration=2.046146938 podStartE2EDuration="2.046146938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:26:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:26:06.044835438 +0000 UTC m=+1235.429340863" watchObservedRunningTime="2026-09-10 22:26:06.046146938 +0000 UTC m=+1235.430652362"
Sep 10 22:26:12.103105 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:12.103066    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:26:15.016411 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:15.016371    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:15.016900 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:15.016456    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:15.029250 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:15.029229    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:15.091199 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:15.091175    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:26:30.598644 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:30.598565    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:26:31.254139 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:31.254101    2563 scope.go:117] "RemoveContainer" containerID="6adf7ae176ee4554218880d5a59e70bcdf4490f4eddce0ba926071d8e00bcbc7"
Sep 10 22:26:44.491797 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.491765    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"]
Sep 10 22:26:44.492410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.492066    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq" podUID="4618db54-bb56-4607-8a88-6617d2abf204" containerName="payload-processing" containerID="cri-o://6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63" gracePeriod=30
Sep 10 22:26:44.744472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.744388    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:26:44.895293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.895262    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4618db54-bb56-4607-8a88-6617d2abf204-plugins-config-volume\") pod \"4618db54-bb56-4607-8a88-6617d2abf204\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") "
Sep 10 22:26:44.895293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.895296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs4xh\" (UniqueName: \"kubernetes.io/projected/4618db54-bb56-4607-8a88-6617d2abf204-kube-api-access-xs4xh\") pod \"4618db54-bb56-4607-8a88-6617d2abf204\" (UID: \"4618db54-bb56-4607-8a88-6617d2abf204\") "
Sep 10 22:26:44.895684 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.895661    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4618db54-bb56-4607-8a88-6617d2abf204-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4618db54-bb56-4607-8a88-6617d2abf204" (UID: "4618db54-bb56-4607-8a88-6617d2abf204"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:26:44.895741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.895728    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4618db54-bb56-4607-8a88-6617d2abf204-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:26:44.897563 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.897538    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4618db54-bb56-4607-8a88-6617d2abf204-kube-api-access-xs4xh" (OuterVolumeSpecName: "kube-api-access-xs4xh") pod "4618db54-bb56-4607-8a88-6617d2abf204" (UID: "4618db54-bb56-4607-8a88-6617d2abf204"). InnerVolumeSpecName "kube-api-access-xs4xh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:26:44.996408 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:44.996338    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs4xh\" (UniqueName: \"kubernetes.io/projected/4618db54-bb56-4607-8a88-6617d2abf204-kube-api-access-xs4xh\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:26:45.230002 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.229965    2563 generic.go:356] "Generic (PLEG): container finished" podID="4618db54-bb56-4607-8a88-6617d2abf204" containerID="6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63" exitCode=0
Sep 10 22:26:45.230172 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.230013    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"
Sep 10 22:26:45.230172 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.230019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq" event={"ID":"4618db54-bb56-4607-8a88-6617d2abf204","Type":"ContainerDied","Data":"6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63"}
Sep 10 22:26:45.230172 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.230071    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq" event={"ID":"4618db54-bb56-4607-8a88-6617d2abf204","Type":"ContainerDied","Data":"458adc4b31d65eea85b680a3693641ce41ab1ff9968dcfbb0c66e7b8e46f97d5"}
Sep 10 22:26:45.230172 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.230098    2563 scope.go:117] "RemoveContainer" containerID="6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63"
Sep 10 22:26:45.238932 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.238911    2563 scope.go:117] "RemoveContainer" containerID="6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63"
Sep 10 22:26:45.239218 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:26:45.239198    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63\": container with ID starting with 6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63 not found: ID does not exist" containerID="6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63"
Sep 10 22:26:45.239295 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.239226    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63"} err="failed to get container status \"6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63\": rpc error: code = NotFound desc = could not find container \"6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63\": container with ID starting with 6a775dd8300cfc246cda6ca4f2fe3dc0126c000c2ef0e95fbe6b8164be4cdb63 not found: ID does not exist"
Sep 10 22:26:45.244762 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.244740    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"]
Sep 10 22:26:45.250043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:45.249992    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-853c9ae4-7d6c6c98b5-c5hkq"]
Sep 10 22:26:47.110616 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:47.110580    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4618db54-bb56-4607-8a88-6617d2abf204" path="/var/lib/kubelet/pods/4618db54-bb56-4607-8a88-6617d2abf204/volumes"
Sep 10 22:26:49.098833 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:49.098796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:26:52.771768 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:26:52.771738    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"]
Sep 10 22:27:06.298977 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:06.298944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:27:07.791274 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:07.791211    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg" podUID="71048100-fdb7-4e6f-b708-958faa9ec6d0" containerName="main" containerID="cri-o://53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32" gracePeriod=45
Sep 10 22:27:08.043323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.043260    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:27:08.105730 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.105702    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-model-cache\") pod \"71048100-fdb7-4e6f-b708-958faa9ec6d0\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") "
Sep 10 22:27:08.105889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.105769    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-dshm\") pod \"71048100-fdb7-4e6f-b708-958faa9ec6d0\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") "
Sep 10 22:27:08.105889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.105790    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-tmp-dir\") pod \"71048100-fdb7-4e6f-b708-958faa9ec6d0\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") "
Sep 10 22:27:08.105889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.105829    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71048100-fdb7-4e6f-b708-958faa9ec6d0-tls-certs\") pod \"71048100-fdb7-4e6f-b708-958faa9ec6d0\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") "
Sep 10 22:27:08.106035 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.105909    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-home\") pod \"71048100-fdb7-4e6f-b708-958faa9ec6d0\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") "
Sep 10 22:27:08.106035 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.105938    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m96lv\" (UniqueName: \"kubernetes.io/projected/71048100-fdb7-4e6f-b708-958faa9ec6d0-kube-api-access-m96lv\") pod \"71048100-fdb7-4e6f-b708-958faa9ec6d0\" (UID: \"71048100-fdb7-4e6f-b708-958faa9ec6d0\") "
Sep 10 22:27:08.106035 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.105988    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "71048100-fdb7-4e6f-b708-958faa9ec6d0" (UID: "71048100-fdb7-4e6f-b708-958faa9ec6d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:27:08.106201 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.106058    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71048100-fdb7-4e6f-b708-958faa9ec6d0" (UID: "71048100-fdb7-4e6f-b708-958faa9ec6d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:27:08.106264 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.106216    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:27:08.106264 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.106228    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-home" (OuterVolumeSpecName: "home") pod "71048100-fdb7-4e6f-b708-958faa9ec6d0" (UID: "71048100-fdb7-4e6f-b708-958faa9ec6d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:27:08.106264 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.106245    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:27:08.108222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.108194    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71048100-fdb7-4e6f-b708-958faa9ec6d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71048100-fdb7-4e6f-b708-958faa9ec6d0" (UID: "71048100-fdb7-4e6f-b708-958faa9ec6d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:27:08.108320 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.108309    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71048100-fdb7-4e6f-b708-958faa9ec6d0-kube-api-access-m96lv" (OuterVolumeSpecName: "kube-api-access-m96lv") pod "71048100-fdb7-4e6f-b708-958faa9ec6d0" (UID: "71048100-fdb7-4e6f-b708-958faa9ec6d0"). InnerVolumeSpecName "kube-api-access-m96lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:27:08.108387 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.108359    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-dshm" (OuterVolumeSpecName: "dshm") pod "71048100-fdb7-4e6f-b708-958faa9ec6d0" (UID: "71048100-fdb7-4e6f-b708-958faa9ec6d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:27:08.207566 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.207527    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:27:08.207566 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.207564    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m96lv\" (UniqueName: \"kubernetes.io/projected/71048100-fdb7-4e6f-b708-958faa9ec6d0-kube-api-access-m96lv\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:27:08.207765 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.207579    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71048100-fdb7-4e6f-b708-958faa9ec6d0-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:27:08.207765 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.207593    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71048100-fdb7-4e6f-b708-958faa9ec6d0-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:27:08.368167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.368083    2563 generic.go:356] "Generic (PLEG): container finished" podID="71048100-fdb7-4e6f-b708-958faa9ec6d0" containerID="53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32" exitCode=0
Sep 10 22:27:08.368167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.368133    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"
Sep 10 22:27:08.368355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.368168    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg" event={"ID":"71048100-fdb7-4e6f-b708-958faa9ec6d0","Type":"ContainerDied","Data":"53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32"}
Sep 10 22:27:08.368355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.368214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg" event={"ID":"71048100-fdb7-4e6f-b708-958faa9ec6d0","Type":"ContainerDied","Data":"e682e0a6c7eb836c705b00bc9381bfec50264a03405383f2310f430067fdb9b6"}
Sep 10 22:27:08.368355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.368230    2563 scope.go:117] "RemoveContainer" containerID="53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32"
Sep 10 22:27:08.377754 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.377731    2563 scope.go:117] "RemoveContainer" containerID="53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32"
Sep 10 22:27:08.378087 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:27:08.378049    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32\": container with ID starting with 53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32 not found: ID does not exist" containerID="53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32"
Sep 10 22:27:08.378187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.378107    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32"} err="failed to get container status \"53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32\": rpc error: code = NotFound desc = could not find container \"53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32\": container with ID starting with 53e192e42e439ee59b8b2d74d75e94a19d1fc5b9a18847d9e22fd7a9c4386c32 not found: ID does not exist"
Sep 10 22:27:08.388863 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.388838    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"]
Sep 10 22:27:08.394414 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:08.394391    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-853c9ae4/e2e-lifecycle-model-853c9ae4-kserve-bd9454cf9-qjtpg"]
Sep 10 22:27:09.111086 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:09.111057    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71048100-fdb7-4e6f-b708-958faa9ec6d0" path="/var/lib/kubelet/pods/71048100-fdb7-4e6f-b708-958faa9ec6d0/volumes"
Sep 10 22:27:22.099219 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:22.099184    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:27:38.097937 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:38.097905    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:27:55.606598 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:55.606512    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"]
Sep 10 22:27:56.694248 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:56.694211    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"]
Sep 10 22:27:57.900968 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:27:57.900935    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:28:10.626277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.626225    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw" podUID="6388d6d1-7612-4b8d-b459-7528fcec4c26" containerName="main" containerID="cri-o://e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628" gracePeriod=45
Sep 10 22:28:10.808697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.808662    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:28:10.875011 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.874985    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:28:10.970741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970653    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-model-cache\") pod \"6388d6d1-7612-4b8d-b459-7528fcec4c26\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") "
Sep 10 22:28:10.970741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970707    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-tmp-dir\") pod \"6388d6d1-7612-4b8d-b459-7528fcec4c26\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") "
Sep 10 22:28:10.970741 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970745    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-home\") pod \"6388d6d1-7612-4b8d-b459-7528fcec4c26\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") "
Sep 10 22:28:10.971006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970790    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz2vv\" (UniqueName: \"kubernetes.io/projected/6388d6d1-7612-4b8d-b459-7528fcec4c26-kube-api-access-bz2vv\") pod \"6388d6d1-7612-4b8d-b459-7528fcec4c26\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") "
Sep 10 22:28:10.971006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970861    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-dshm\") pod \"6388d6d1-7612-4b8d-b459-7528fcec4c26\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") "
Sep 10 22:28:10.971006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970888    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6388d6d1-7612-4b8d-b459-7528fcec4c26-tls-certs\") pod \"6388d6d1-7612-4b8d-b459-7528fcec4c26\" (UID: \"6388d6d1-7612-4b8d-b459-7528fcec4c26\") "
Sep 10 22:28:10.971006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970952    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-model-cache" (OuterVolumeSpecName: "model-cache") pod "6388d6d1-7612-4b8d-b459-7528fcec4c26" (UID: "6388d6d1-7612-4b8d-b459-7528fcec4c26"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:10.971006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.970972    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6388d6d1-7612-4b8d-b459-7528fcec4c26" (UID: "6388d6d1-7612-4b8d-b459-7528fcec4c26"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:10.971260 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.971034    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-home" (OuterVolumeSpecName: "home") pod "6388d6d1-7612-4b8d-b459-7528fcec4c26" (UID: "6388d6d1-7612-4b8d-b459-7528fcec4c26"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:10.971260 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.971163    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:10.971260 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.971174    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:10.971260 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.971182    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:10.973279 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.973252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-dshm" (OuterVolumeSpecName: "dshm") pod "6388d6d1-7612-4b8d-b459-7528fcec4c26" (UID: "6388d6d1-7612-4b8d-b459-7528fcec4c26"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:10.973429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.973383    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6388d6d1-7612-4b8d-b459-7528fcec4c26-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6388d6d1-7612-4b8d-b459-7528fcec4c26" (UID: "6388d6d1-7612-4b8d-b459-7528fcec4c26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:28:10.973520 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:10.973430    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6388d6d1-7612-4b8d-b459-7528fcec4c26-kube-api-access-bz2vv" (OuterVolumeSpecName: "kube-api-access-bz2vv") pod "6388d6d1-7612-4b8d-b459-7528fcec4c26" (UID: "6388d6d1-7612-4b8d-b459-7528fcec4c26"). InnerVolumeSpecName "kube-api-access-bz2vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:28:11.071785 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.071755    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6388d6d1-7612-4b8d-b459-7528fcec4c26-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:11.071785 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.071780    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6388d6d1-7612-4b8d-b459-7528fcec4c26-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:11.071953 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.071795    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz2vv\" (UniqueName: \"kubernetes.io/projected/6388d6d1-7612-4b8d-b459-7528fcec4c26-kube-api-access-bz2vv\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:11.695877 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.695828    2563 generic.go:356] "Generic (PLEG): container finished" podID="6388d6d1-7612-4b8d-b459-7528fcec4c26" containerID="e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628" exitCode=0
Sep 10 22:28:11.695877 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.695872    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"
Sep 10 22:28:11.696410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.695901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw" event={"ID":"6388d6d1-7612-4b8d-b459-7528fcec4c26","Type":"ContainerDied","Data":"e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628"}
Sep 10 22:28:11.696410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.695951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw" event={"ID":"6388d6d1-7612-4b8d-b459-7528fcec4c26","Type":"ContainerDied","Data":"ff5609a514223a98c64ee31b65a7a0975cbd3a780712d69be0fb43181ec02ca8"}
Sep 10 22:28:11.696410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.695974    2563 scope.go:117] "RemoveContainer" containerID="e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628"
Sep 10 22:28:11.704528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.704508    2563 scope.go:117] "RemoveContainer" containerID="e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628"
Sep 10 22:28:11.704855 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:28:11.704832    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628\": container with ID starting with e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628 not found: ID does not exist" containerID="e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628"
Sep 10 22:28:11.704904 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.704869    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628"} err="failed to get container status \"e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628\": rpc error: code = NotFound desc = could not find container \"e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628\": container with ID starting with e1ef869f90d94c15020843a53b5f1f9dacbf92de2183b13ace57ac0359c1e628 not found: ID does not exist"
Sep 10 22:28:11.711451 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.711413    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"]
Sep 10 22:28:11.714312 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.714285    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5" podUID="9434136e-f753-480e-bb0b-15f823ceb6a7" containerName="main" containerID="cri-o://3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681" gracePeriod=45
Sep 10 22:28:11.715614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.715593    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/auth-test-model-e367c1-kserve-579569b675-hcglw"]
Sep 10 22:28:11.959132 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:11.959070    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:28:12.079720 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.079686    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-model-cache\") pod \"9434136e-f753-480e-bb0b-15f823ceb6a7\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") "
Sep 10 22:28:12.079882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.079774    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-dshm\") pod \"9434136e-f753-480e-bb0b-15f823ceb6a7\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") "
Sep 10 22:28:12.079882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.079825    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-tmp-dir\") pod \"9434136e-f753-480e-bb0b-15f823ceb6a7\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") "
Sep 10 22:28:12.079882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.079857    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9434136e-f753-480e-bb0b-15f823ceb6a7-tls-certs\") pod \"9434136e-f753-480e-bb0b-15f823ceb6a7\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") "
Sep 10 22:28:12.080008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.079918    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxxfb\" (UniqueName: \"kubernetes.io/projected/9434136e-f753-480e-bb0b-15f823ceb6a7-kube-api-access-gxxfb\") pod \"9434136e-f753-480e-bb0b-15f823ceb6a7\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") "
Sep 10 22:28:12.080008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.079943    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-home\") pod \"9434136e-f753-480e-bb0b-15f823ceb6a7\" (UID: \"9434136e-f753-480e-bb0b-15f823ceb6a7\") "
Sep 10 22:28:12.080008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.079998    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "9434136e-f753-480e-bb0b-15f823ceb6a7" (UID: "9434136e-f753-480e-bb0b-15f823ceb6a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:12.080163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.080129    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9434136e-f753-480e-bb0b-15f823ceb6a7" (UID: "9434136e-f753-480e-bb0b-15f823ceb6a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:12.080286 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.080258    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-home" (OuterVolumeSpecName: "home") pod "9434136e-f753-480e-bb0b-15f823ceb6a7" (UID: "9434136e-f753-480e-bb0b-15f823ceb6a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:12.080408 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.080268    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:12.080408 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.080320    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:12.082237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.082208    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9434136e-f753-480e-bb0b-15f823ceb6a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9434136e-f753-480e-bb0b-15f823ceb6a7" (UID: "9434136e-f753-480e-bb0b-15f823ceb6a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:28:12.082329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.082232    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-dshm" (OuterVolumeSpecName: "dshm") pod "9434136e-f753-480e-bb0b-15f823ceb6a7" (UID: "9434136e-f753-480e-bb0b-15f823ceb6a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:28:12.082329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.082244    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9434136e-f753-480e-bb0b-15f823ceb6a7-kube-api-access-gxxfb" (OuterVolumeSpecName: "kube-api-access-gxxfb") pod "9434136e-f753-480e-bb0b-15f823ceb6a7" (UID: "9434136e-f753-480e-bb0b-15f823ceb6a7"). InnerVolumeSpecName "kube-api-access-gxxfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:28:12.181732 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.181698    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9434136e-f753-480e-bb0b-15f823ceb6a7-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:12.181732 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.181725    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxxfb\" (UniqueName: \"kubernetes.io/projected/9434136e-f753-480e-bb0b-15f823ceb6a7-kube-api-access-gxxfb\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:12.181732 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.181734    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:12.181732 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.181743    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9434136e-f753-480e-bb0b-15f823ceb6a7-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:28:12.703426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.703392    2563 generic.go:356] "Generic (PLEG): container finished" podID="9434136e-f753-480e-bb0b-15f823ceb6a7" containerID="3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681" exitCode=0
Sep 10 22:28:12.703855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.703454    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"
Sep 10 22:28:12.703855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.703481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5" event={"ID":"9434136e-f753-480e-bb0b-15f823ceb6a7","Type":"ContainerDied","Data":"3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681"}
Sep 10 22:28:12.703855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.703526    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5" event={"ID":"9434136e-f753-480e-bb0b-15f823ceb6a7","Type":"ContainerDied","Data":"09421e792ced9f741865be48ab829d962ba4a38bae5b823db005666408f07b8a"}
Sep 10 22:28:12.703855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.703545    2563 scope.go:117] "RemoveContainer" containerID="3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681"
Sep 10 22:28:12.714740 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.714721    2563 scope.go:117] "RemoveContainer" containerID="3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681"
Sep 10 22:28:12.715028 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:28:12.715007    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681\": container with ID starting with 3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681 not found: ID does not exist" containerID="3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681"
Sep 10 22:28:12.715101 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.715041    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681"} err="failed to get container status \"3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681\": rpc error: code = NotFound desc = could not find container \"3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681\": container with ID starting with 3b56263e667587dbd627d8675509564fa9f9c7f1416f757f0c91480f6ffa0681 not found: ID does not exist"
Sep 10 22:28:12.723256 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.723223    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"]
Sep 10 22:28:12.725361 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:12.725342    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/auth-test-model-0ada08-kserve-7cc57d9c5f-s7vs5"]
Sep 10 22:28:13.111024 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:13.110989    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6388d6d1-7612-4b8d-b459-7528fcec4c26" path="/var/lib/kubelet/pods/6388d6d1-7612-4b8d-b459-7528fcec4c26/volumes"
Sep 10 22:28:13.111375 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:13.111363    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9434136e-f753-480e-bb0b-15f823ceb6a7" path="/var/lib/kubelet/pods/9434136e-f753-480e-bb0b-15f823ceb6a7/volumes"
Sep 10 22:28:25.110897 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.110870    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"]
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111206    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71048100-fdb7-4e6f-b708-958faa9ec6d0" containerName="main"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111215    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71048100-fdb7-4e6f-b708-958faa9ec6d0" containerName="main"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111227    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9434136e-f753-480e-bb0b-15f823ceb6a7" containerName="main"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111232    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9434136e-f753-480e-bb0b-15f823ceb6a7" containerName="main"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111246    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6388d6d1-7612-4b8d-b459-7528fcec4c26" containerName="main"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111251    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6388d6d1-7612-4b8d-b459-7528fcec4c26" containerName="main"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111257    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4618db54-bb56-4607-8a88-6617d2abf204" containerName="payload-processing"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111262    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4618db54-bb56-4607-8a88-6617d2abf204" containerName="payload-processing"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111333    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6388d6d1-7612-4b8d-b459-7528fcec4c26" containerName="main"
Sep 10 22:28:25.111338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111341    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71048100-fdb7-4e6f-b708-958faa9ec6d0" containerName="main"
Sep 10 22:28:25.111678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111347    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4618db54-bb56-4607-8a88-6617d2abf204" containerName="payload-processing"
Sep 10 22:28:25.111678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.111354    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9434136e-f753-480e-bb0b-15f823ceb6a7" containerName="main"
Sep 10 22:28:25.115582 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.115566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.118646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.118616    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"sub-test-model-0ada08-kserve-self-signed-certs\""
Sep 10 22:28:25.118646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.118638    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"default-dockercfg-567d7\""
Sep 10 22:28:25.118921 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.118679    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"kube-root-ca.crt\""
Sep 10 22:28:25.118921 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.118829    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"openshift-service-ca.crt\""
Sep 10 22:28:25.119338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.119316    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"]
Sep 10 22:28:25.201625 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.201580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhzf5\" (UniqueName: \"kubernetes.io/projected/8d642b79-eff6-45ce-9907-a4e807d57144-kube-api-access-lhzf5\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.201812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.201643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-tmp-dir\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.201812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.201773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-model-cache\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.201912 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.201865    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-home\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.201969 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.201919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d642b79-eff6-45ce-9907-a4e807d57144-tls-certs\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.202029 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.202014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-dshm\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.303299 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303260    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhzf5\" (UniqueName: \"kubernetes.io/projected/8d642b79-eff6-45ce-9907-a4e807d57144-kube-api-access-lhzf5\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.303534 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-tmp-dir\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.303534 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-model-cache\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.303534 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303409    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-home\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.303534 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d642b79-eff6-45ce-9907-a4e807d57144-tls-certs\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.303534 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-dshm\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.303872 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303832    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-model-cache\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.304015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-home\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.304015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.303886    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-tmp-dir\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.305993 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.305971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-dshm\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.306107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.306083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d642b79-eff6-45ce-9907-a4e807d57144-tls-certs\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.310867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.310838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhzf5\" (UniqueName: \"kubernetes.io/projected/8d642b79-eff6-45ce-9907-a4e807d57144-kube-api-access-lhzf5\") pod \"sub-test-model-0ada08-kserve-67c5f58455-6nmmd\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.426296 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.426202    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:25.697194 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.697170    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"]
Sep 10 22:28:25.699313 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:28:25.699277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d642b79_eff6_45ce_9907_a4e807d57144.slice/crio-c4f03ba63189de074f777256b38abde2dc2d9cbe0544fdd2bc2335e294346c47 WatchSource:0}: Error finding container c4f03ba63189de074f777256b38abde2dc2d9cbe0544fdd2bc2335e294346c47: Status 404 returned error can't find the container with id c4f03ba63189de074f777256b38abde2dc2d9cbe0544fdd2bc2335e294346c47
Sep 10 22:28:25.772145 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.772112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd" event={"ID":"8d642b79-eff6-45ce-9907-a4e807d57144","Type":"ContainerStarted","Data":"ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536"}
Sep 10 22:28:25.772253 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.772159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd" event={"ID":"8d642b79-eff6-45ce-9907-a4e807d57144","Type":"ContainerStarted","Data":"c4f03ba63189de074f777256b38abde2dc2d9cbe0544fdd2bc2335e294346c47"}
Sep 10 22:28:25.790714 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:25.790666    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd" podStartSLOduration=0.790648043 podStartE2EDuration="790.648043ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:28:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:28:25.789528748 +0000 UTC m=+1375.174034174" watchObservedRunningTime="2026-09-10 22:28:25.790648043 +0000 UTC m=+1375.175153470"
Sep 10 22:28:29.807372 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:29.807338    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:28:35.426914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.426863    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:35.426914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.426922    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:35.439671 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.439645    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:35.527310 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.527277    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"]
Sep 10 22:28:35.532289 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.532267    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.534539 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.534507    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-19922b-openshift-default-dockercfg-jcmsq\""
Sep 10 22:28:35.541744 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.541724    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"]
Sep 10 22:28:35.696080 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.695994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-token\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696080 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-socket\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696080 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-data\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-podinfo\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-envoy\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-credential-socket\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696199    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt7g7\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-kube-api-access-kt7g7\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istiod-ca-cert\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.696365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.696259    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-certs\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-data\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-podinfo\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-envoy\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-credential-socket\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt7g7\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-kube-api-access-kt7g7\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797338 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istiod-ca-cert\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-certs\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-token\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797421    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-socket\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-data\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797857 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797762    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-certs\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.797857 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.797803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-socket\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.798193 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.798173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-credential-socket\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.798312 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.798288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istiod-ca-cert\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.799738 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.799715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-envoy\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.799808 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.799791    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-podinfo\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.805340 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.805316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-token\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.805588 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.805570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt7g7\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-kube-api-access-kt7g7\") pod \"e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") " pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:35.832786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.832763    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:28:35.845808 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:35.845780    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:36.123226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.123192    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"]
Sep 10 22:28:36.124644 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:28:36.124621    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d34a01c_bf0e_4013_b47c_9b4ab0f695c7.slice/crio-d6b9e972366d2a9ea0799254bb13fd08a6071518c10a08091ddcf2c1c8c83cad WatchSource:0}: Error finding container d6b9e972366d2a9ea0799254bb13fd08a6071518c10a08091ddcf2c1c8c83cad: Status 404 returned error can't find the container with id d6b9e972366d2a9ea0799254bb13fd08a6071518c10a08091ddcf2c1c8c83cad
Sep 10 22:28:36.127046 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.127006    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:28:36.127129 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.127092    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:28:36.127169 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.127132    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:28:36.828068 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.828035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" event={"ID":"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7","Type":"ContainerStarted","Data":"548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1"}
Sep 10 22:28:36.828570 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.828081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" event={"ID":"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7","Type":"ContainerStarted","Data":"d6b9e972366d2a9ea0799254bb13fd08a6071518c10a08091ddcf2c1c8c83cad"}
Sep 10 22:28:36.846200 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.846169    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:36.853357 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:36.853300    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podStartSLOduration=1.8532796569999999 podStartE2EDuration="1.853279657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:28:36.849684934 +0000 UTC m=+1386.234190358" watchObservedRunningTime="2026-09-10 22:28:36.853279657 +0000 UTC m=+1386.237785084"
Sep 10 22:28:37.852406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:37.852376    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:38.838416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:38.838382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:38.839270 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:38.839253    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:28:48.406640 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:48.406611    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:28:59.908489 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:28:59.908450    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:29:04.445846 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.445808    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"]
Sep 10 22:29:04.449960 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.449939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.452213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.452191    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-ad8659-openshift-default-dockercfg-x9zxd\""
Sep 10 22:29:04.459416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.459394    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"]
Sep 10 22:29:04.559917 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.559774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-data\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.559917 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.559891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-certs\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.560197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.559925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrx7q\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-kube-api-access-hrx7q\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.560197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.559972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-socket\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.560197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.560016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-podinfo\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.560197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.560082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-envoy\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.560197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.560104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-credential-socket\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.560197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.560139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-token\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.560197 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.560163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7634911c-2073-4b6d-a8e0-af7fe7167434-istiod-ca-cert\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661207 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-socket\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661207 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-podinfo\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-envoy\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-credential-socket\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-token\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7634911c-2073-4b6d-a8e0-af7fe7167434-istiod-ca-cert\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-data\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661491 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-certs\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661814 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrx7q\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-kube-api-access-hrx7q\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.661814 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-socket\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.662022 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.661984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-data\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.662157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.662042    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7634911c-2073-4b6d-a8e0-af7fe7167434-istiod-ca-cert\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.662219 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.662156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-credential-socket\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.662307 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.662293    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-certs\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.663849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.663827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-envoy\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.663963 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.663946    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-podinfo\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.669867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.669846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-token\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.669968 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.669908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrx7q\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-kube-api-access-hrx7q\") pod \"e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") " pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:04.764932 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:04.764897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:05.071343 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:05.071315    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"]
Sep 10 22:29:05.074060 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:29:05.074035    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7634911c_2073_4b6d_a8e0_af7fe7167434.slice/crio-8438ced260f79c9a9f59fd71dc5bc33f6baacff40a85fbc243d1ee7c7a5be996 WatchSource:0}: Error finding container 8438ced260f79c9a9f59fd71dc5bc33f6baacff40a85fbc243d1ee7c7a5be996: Status 404 returned error can't find the container with id 8438ced260f79c9a9f59fd71dc5bc33f6baacff40a85fbc243d1ee7c7a5be996
Sep 10 22:29:05.076887 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:05.076854    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:05.076966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:05.076917    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:05.076966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:05.076953    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:05.976270 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:05.976229    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" event={"ID":"7634911c-2073-4b6d-a8e0-af7fe7167434","Type":"ContainerStarted","Data":"2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e"}
Sep 10 22:29:05.976765 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:05.976309    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" event={"ID":"7634911c-2073-4b6d-a8e0-af7fe7167434","Type":"ContainerStarted","Data":"8438ced260f79c9a9f59fd71dc5bc33f6baacff40a85fbc243d1ee7c7a5be996"}
Sep 10 22:29:05.999715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:05.999667    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podStartSLOduration=1.999652864 podStartE2EDuration="1.999652864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:29:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:29:05.997603383 +0000 UTC m=+1415.382108809" watchObservedRunningTime="2026-09-10 22:29:05.999652864 +0000 UTC m=+1415.384158289"
Sep 10 22:29:06.765419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:06.765380    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:06.770856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:06.770830    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:06.981056 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:06.981017    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:06.981851 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:06.981831    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:09.838357 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.838276    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"]
Sep 10 22:29:09.842181 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.842159    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:09.844851 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.844825    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-ad8659-dockercfg-4hm6g\""
Sep 10 22:29:09.844967 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.844876    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-ad8659\""
Sep 10 22:29:09.858071 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.858048    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"]
Sep 10 22:29:09.880662 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.880628    2563 patch_prober.go:28] interesting pod/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 22:29:09.880782 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.880710    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 22:29:09.914074 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.914039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43b75795-5d05-45dc-95b5-2b97c5c78657-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") " pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:09.914277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:09.914101    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtxt7\" (UniqueName: \"kubernetes.io/projected/43b75795-5d05-45dc-95b5-2b97c5c78657-kube-api-access-mtxt7\") pod \"payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") " pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:10.015742 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:10.015711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43b75795-5d05-45dc-95b5-2b97c5c78657-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") " pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:10.015863 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:10.015769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtxt7\" (UniqueName: \"kubernetes.io/projected/43b75795-5d05-45dc-95b5-2b97c5c78657-kube-api-access-mtxt7\") pod \"payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") " pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:10.016611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:10.016577    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43b75795-5d05-45dc-95b5-2b97c5c78657-plugins-config-volume\") pod \"payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") " pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:10.023966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:10.023943    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtxt7\" (UniqueName: \"kubernetes.io/projected/43b75795-5d05-45dc-95b5-2b97c5c78657-kube-api-access-mtxt7\") pod \"payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") " pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:10.154034 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:10.153957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:10.422144 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:10.421984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"]
Sep 10 22:29:10.425098 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:29:10.425069    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b75795_5d05_45dc_95b5_2b97c5c78657.slice/crio-22e1a5128302da0081b8836e7755593bfc26e83f3ece547de21750454317c924 WatchSource:0}: Error finding container 22e1a5128302da0081b8836e7755593bfc26e83f3ece547de21750454317c924: Status 404 returned error can't find the container with id 22e1a5128302da0081b8836e7755593bfc26e83f3ece547de21750454317c924
Sep 10 22:29:11.004693 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.004665    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerID="548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1" exitCode=0
Sep 10 22:29:11.005187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.004746    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" event={"ID":"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7","Type":"ContainerDied","Data":"548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1"}
Sep 10 22:29:11.005267 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.005219    2563 scope.go:117] "RemoveContainer" containerID="548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1"
Sep 10 22:29:11.006522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.006501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8" event={"ID":"43b75795-5d05-45dc-95b5-2b97c5c78657","Type":"ContainerStarted","Data":"c2f20f8e839042c558c6fe85e5c50bc1296ad1716bc3cd9a3f98d3297bbc7628"}
Sep 10 22:29:11.006638 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.006564    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:11.006638 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.006599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8" event={"ID":"43b75795-5d05-45dc-95b5-2b97c5c78657","Type":"ContainerStarted","Data":"22e1a5128302da0081b8836e7755593bfc26e83f3ece547de21750454317c924"}
Sep 10 22:29:11.006638 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.006624    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:11.006805 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.006682    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:11.006805 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.006714    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:11.044112 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:11.044057    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8" podStartSLOduration=2.044033238 podStartE2EDuration="2.044033238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:29:11.041635702 +0000 UTC m=+1420.426141129" watchObservedRunningTime="2026-09-10 22:29:11.044033238 +0000 UTC m=+1420.428538664"
Sep 10 22:29:12.018218 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:12.018169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" event={"ID":"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7","Type":"ContainerStarted","Data":"b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2"}
Sep 10 22:29:12.846167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:12.846127    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:29:12.846370 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:12.846332    2563 patch_prober.go:28] interesting pod/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 22:29:12.846495 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:12.846402    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 22:29:13.846978 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:13.846942    2563 patch_prober.go:28] interesting pod/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 22:29:13.847386 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:13.847010    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 22:29:14.846914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:14.846857    2563 patch_prober.go:28] interesting pod/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 22:29:14.847112 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:14.846933    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 22:29:15.846569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:15.846535    2563 patch_prober.go:28] interesting pod/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 22:29:15.846815 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:15.846792    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 22:29:16.846543 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:16.846501    2563 patch_prober.go:28] interesting pod/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 10 22:29:16.846950 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:16.846561    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 10 22:29:17.850527 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:17.850497    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:29:17.850993 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:17.850904    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:29:17.851676 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:17.851660    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:29:20.788926 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:20.788895    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"]
Sep 10 22:29:22.021036 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:22.020964    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:29:22.111806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:22.109206    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:29:29.703330 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.703293    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"]
Sep 10 22:29:29.706771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.706746    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.713947 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.713580    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"openshift-service-ca.crt\""
Sep 10 22:29:29.713947 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.713600    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"default-dockercfg-rxtzj\""
Sep 10 22:29:29.713947 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.713608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"rate-test-model-e367c1-kserve-self-signed-certs\""
Sep 10 22:29:29.713947 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.713652    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-e367c1\"/\"kube-root-ca.crt\""
Sep 10 22:29:29.722989 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.722965    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"]
Sep 10 22:29:29.802037 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.801959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-home\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.802037 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.802009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2zbs\" (UniqueName: \"kubernetes.io/projected/403d9185-479c-4070-94e1-9860cac88819-kube-api-access-n2zbs\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.802217 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.802056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-model-cache\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.802217 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.802147    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403d9185-479c-4070-94e1-9860cac88819-tls-certs\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.802287 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.802219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-tmp-dir\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.802287 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.802256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-dshm\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903343 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903303    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2zbs\" (UniqueName: \"kubernetes.io/projected/403d9185-479c-4070-94e1-9860cac88819-kube-api-access-n2zbs\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903373    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-model-cache\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403d9185-479c-4070-94e1-9860cac88819-tls-certs\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-tmp-dir\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-dshm\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903569 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-home\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-model-cache\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-tmp-dir\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.903953 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.903939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-home\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.905985 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.905956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-dshm\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.906221 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.906198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403d9185-479c-4070-94e1-9860cac88819-tls-certs\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:29.911820 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:29.911800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2zbs\" (UniqueName: \"kubernetes.io/projected/403d9185-479c-4070-94e1-9860cac88819-kube-api-access-n2zbs\") pod \"rate-test-model-e367c1-kserve-86d8b6977f-zqn74\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") " pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:30.016151 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:30.016102    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:30.292867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:30.292841    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"]
Sep 10 22:29:30.295387 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:29:30.295344    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod403d9185_479c_4070_94e1_9860cac88819.slice/crio-3a51dcee08ff56ee1b02d41d13dba18be268c0cd22542bfb642ce0d22b54d67e WatchSource:0}: Error finding container 3a51dcee08ff56ee1b02d41d13dba18be268c0cd22542bfb642ce0d22b54d67e: Status 404 returned error can't find the container with id 3a51dcee08ff56ee1b02d41d13dba18be268c0cd22542bfb642ce0d22b54d67e
Sep 10 22:29:31.152751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:31.152714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74" event={"ID":"403d9185-479c-4070-94e1-9860cac88819","Type":"ContainerStarted","Data":"1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b"}
Sep 10 22:29:31.153228 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:31.152764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74" event={"ID":"403d9185-479c-4070-94e1-9860cac88819","Type":"ContainerStarted","Data":"3a51dcee08ff56ee1b02d41d13dba18be268c0cd22542bfb642ce0d22b54d67e"}
Sep 10 22:29:31.167840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:31.167790    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74" podStartSLOduration=2.167771633 podStartE2EDuration="2.167771633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:29:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:29:31.166647656 +0000 UTC m=+1440.551153084" watchObservedRunningTime="2026-09-10 22:29:31.167771633 +0000 UTC m=+1440.552277057"
Sep 10 22:29:33.163813 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:33.163783    2563 generic.go:356] "Generic (PLEG): container finished" podID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerID="2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e" exitCode=0
Sep 10 22:29:33.164237 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:33.163861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" event={"ID":"7634911c-2073-4b6d-a8e0-af7fe7167434","Type":"ContainerDied","Data":"2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e"}
Sep 10 22:29:33.164283 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:33.164272    2563 scope.go:117] "RemoveContainer" containerID="2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e"
Sep 10 22:29:33.165397 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:33.165368    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:33.165528 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:33.165479    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:33.165594 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:33.165531    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:29:34.170835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:34.170800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" event={"ID":"7634911c-2073-4b6d-a8e0-af7fe7167434","Type":"ContainerStarted","Data":"59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781"}
Sep 10 22:29:34.765386 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:34.765349    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:34.766385 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:34.766357    2563 patch_prober.go:28] interesting pod/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 10 22:29:34.766527 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:34.766431    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 10 22:29:35.765813 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:35.765769    2563 patch_prober.go:28] interesting pod/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 10 22:29:35.766201 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:35.765849    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 10 22:29:35.808485 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:35.808428    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd" podUID="8d642b79-eff6-45ce-9907-a4e807d57144" containerName="main" containerID="cri-o://ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536" gracePeriod=45
Sep 10 22:29:35.822523 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:35.822197    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd" podUID="8d642b79-eff6-45ce-9907-a4e807d57144" containerName="main" probeResult="failure" output="Get \"https://10.134.0.76:8000/ready\": dial tcp 10.134.0.76:8000: connect: connection refused"
Sep 10 22:29:36.126991 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.126637    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:29:36.158416 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158386    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-model-cache\") pod \"8d642b79-eff6-45ce-9907-a4e807d57144\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") "
Sep 10 22:29:36.158615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158453    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d642b79-eff6-45ce-9907-a4e807d57144-tls-certs\") pod \"8d642b79-eff6-45ce-9907-a4e807d57144\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") "
Sep 10 22:29:36.158615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158476    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-tmp-dir\") pod \"8d642b79-eff6-45ce-9907-a4e807d57144\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") "
Sep 10 22:29:36.158615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158492    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-home\") pod \"8d642b79-eff6-45ce-9907-a4e807d57144\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") "
Sep 10 22:29:36.158615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158516    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-dshm\") pod \"8d642b79-eff6-45ce-9907-a4e807d57144\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") "
Sep 10 22:29:36.158615 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158616    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhzf5\" (UniqueName: \"kubernetes.io/projected/8d642b79-eff6-45ce-9907-a4e807d57144-kube-api-access-lhzf5\") pod \"8d642b79-eff6-45ce-9907-a4e807d57144\" (UID: \"8d642b79-eff6-45ce-9907-a4e807d57144\") "
Sep 10 22:29:36.158902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158728    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8d642b79-eff6-45ce-9907-a4e807d57144" (UID: "8d642b79-eff6-45ce-9907-a4e807d57144"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:29:36.158902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158713    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-model-cache" (OuterVolumeSpecName: "model-cache") pod "8d642b79-eff6-45ce-9907-a4e807d57144" (UID: "8d642b79-eff6-45ce-9907-a4e807d57144"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:29:36.158902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158751    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-home" (OuterVolumeSpecName: "home") pod "8d642b79-eff6-45ce-9907-a4e807d57144" (UID: "8d642b79-eff6-45ce-9907-a4e807d57144"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:29:36.158902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158897    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:29:36.159118 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158915    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:29:36.159118 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.158928    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:29:36.160801 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.160782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d642b79-eff6-45ce-9907-a4e807d57144-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8d642b79-eff6-45ce-9907-a4e807d57144" (UID: "8d642b79-eff6-45ce-9907-a4e807d57144"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:29:36.162051 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.162017    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d642b79-eff6-45ce-9907-a4e807d57144-kube-api-access-lhzf5" (OuterVolumeSpecName: "kube-api-access-lhzf5") pod "8d642b79-eff6-45ce-9907-a4e807d57144" (UID: "8d642b79-eff6-45ce-9907-a4e807d57144"). InnerVolumeSpecName "kube-api-access-lhzf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:29:36.162051 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.162040    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-dshm" (OuterVolumeSpecName: "dshm") pod "8d642b79-eff6-45ce-9907-a4e807d57144" (UID: "8d642b79-eff6-45ce-9907-a4e807d57144"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:29:36.182417 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.182394    2563 generic.go:356] "Generic (PLEG): container finished" podID="8d642b79-eff6-45ce-9907-a4e807d57144" containerID="ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536" exitCode=0
Sep 10 22:29:36.182552 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.182448    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"
Sep 10 22:29:36.182552 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.182469    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd" event={"ID":"8d642b79-eff6-45ce-9907-a4e807d57144","Type":"ContainerDied","Data":"ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536"}
Sep 10 22:29:36.182552 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.182519    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd" event={"ID":"8d642b79-eff6-45ce-9907-a4e807d57144","Type":"ContainerDied","Data":"c4f03ba63189de074f777256b38abde2dc2d9cbe0544fdd2bc2335e294346c47"}
Sep 10 22:29:36.182552 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.182542    2563 scope.go:117] "RemoveContainer" containerID="ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536"
Sep 10 22:29:36.191342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.191326    2563 scope.go:117] "RemoveContainer" containerID="ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536"
Sep 10 22:29:36.191657 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:29:36.191635    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536\": container with ID starting with ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536 not found: ID does not exist" containerID="ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536"
Sep 10 22:29:36.191714 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.191667    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536"} err="failed to get container status \"ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536\": rpc error: code = NotFound desc = could not find container \"ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536\": container with ID starting with ee2b214095d484b3686eed3087dd1f51697d8b475f12a916153a641186799536 not found: ID does not exist"
Sep 10 22:29:36.203453 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.203410    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"]
Sep 10 22:29:36.206974 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.206954    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/sub-test-model-0ada08-kserve-67c5f58455-6nmmd"]
Sep 10 22:29:36.259705 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.259678    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d642b79-eff6-45ce-9907-a4e807d57144-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:29:36.259705 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.259700    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d642b79-eff6-45ce-9907-a4e807d57144-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:29:36.259705 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.259709    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhzf5\" (UniqueName: \"kubernetes.io/projected/8d642b79-eff6-45ce-9907-a4e807d57144-kube-api-access-lhzf5\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:29:36.765452 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.765399    2563 patch_prober.go:28] interesting pod/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 10 22:29:36.765633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.765488    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 10 22:29:36.981094 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:36.981065    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:37.110881 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:37.110802    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d642b79-eff6-45ce-9907-a4e807d57144" path="/var/lib/kubelet/pods/8d642b79-eff6-45ce-9907-a4e807d57144/volumes"
Sep 10 22:29:37.765676 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:37.765643    2563 patch_prober.go:28] interesting pod/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 10 22:29:37.765859 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:37.765700    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 10 22:29:38.766102 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:38.766059    2563 patch_prober.go:28] interesting pod/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Sep 10 22:29:38.766526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:38.766127    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Sep 10 22:29:39.800073 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:39.800036    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:39.801432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:39.801403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:29:40.016624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:40.016587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:40.016898 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:40.016865    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:40.029253 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:40.029233    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:40.215872 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:40.215792    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:29:44.313738 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:44.313707    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:29:58.407688 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:58.407651    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"]
Sep 10 22:29:58.408281 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:29:58.407934    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" containerID="cri-o://b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2" gracePeriod=30
Sep 10 22:30:00.016341 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.016299    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:30:00.175129 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.175097    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"]
Sep 10 22:30:00.175573 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.175557    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d642b79-eff6-45ce-9907-a4e807d57144" containerName="main"
Sep 10 22:30:00.175624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.175575    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d642b79-eff6-45ce-9907-a4e807d57144" containerName="main"
Sep 10 22:30:00.175681 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.175671    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d642b79-eff6-45ce-9907-a4e807d57144" containerName="main"
Sep 10 22:30:00.178678 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.178663    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"
Sep 10 22:30:00.181367 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.181347    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlfhj\""
Sep 10 22:30:00.183902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.183883    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"]
Sep 10 22:30:00.187309 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.187286    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"
Sep 10 22:30:00.189046 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.189027    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"]
Sep 10 22:30:00.189961 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.189942    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"
Sep 10 22:30:00.193179 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.192992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"]
Sep 10 22:30:00.193179 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.193134    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"
Sep 10 22:30:00.205824 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.205796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"
Sep 10 22:30:00.205999 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.205983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"]
Sep 10 22:30:00.210817 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.210011    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"]
Sep 10 22:30:00.249629 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.249605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"
Sep 10 22:30:00.490698 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.487469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"]
Sep 10 22:30:00.495221 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.495197    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"]
Sep 10 22:30:00.496472 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:30:00.496417    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f37169d_7606_4139_8a27_8b6b4b671ac0.slice/crio-2de3ee99fcc602905d8b7f9142b23d6702b045c4f5dd184e0e329929b62f5310 WatchSource:0}: Error finding container 2de3ee99fcc602905d8b7f9142b23d6702b045c4f5dd184e0e329929b62f5310: Status 404 returned error can't find the container with id 2de3ee99fcc602905d8b7f9142b23d6702b045c4f5dd184e0e329929b62f5310
Sep 10 22:30:00.546053 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:00.546029    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"]
Sep 10 22:30:00.547953 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:30:00.547926    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode259aded_5c9c_4fcc_85fa_4dfe9f8fad4d.slice/crio-de63c58a3d86b9a5a75550bd72cbb61d8a735c3472a97d279ec742aa5405ffaf WatchSource:0}: Error finding container de63c58a3d86b9a5a75550bd72cbb61d8a735c3472a97d279ec742aa5405ffaf: Status 404 returned error can't find the container with id de63c58a3d86b9a5a75550bd72cbb61d8a735c3472a97d279ec742aa5405ffaf
Sep 10 22:30:01.323378 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.323348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2" event={"ID":"e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d","Type":"ContainerStarted","Data":"d8ec5d74a93e34a505c33524c4ada10b3c0e6484faa9ce5c21fe697b7aef769e"}
Sep 10 22:30:01.323821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.323405    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2" event={"ID":"e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d","Type":"ContainerStarted","Data":"de63c58a3d86b9a5a75550bd72cbb61d8a735c3472a97d279ec742aa5405ffaf"}
Sep 10 22:30:01.325023 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.324997    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8" event={"ID":"f21aaea6-7184-46eb-a2d3-00050465c831","Type":"ContainerStarted","Data":"aa7979f86811b8e30531914f2a48d2b063173b92424892a6eaa802da8c1913c3"}
Sep 10 22:30:01.325155 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.325054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8" event={"ID":"f21aaea6-7184-46eb-a2d3-00050465c831","Type":"ContainerStarted","Data":"082bc498f577ac61f992bed987c762ef309ba529c0747b1fded21b519edeb90e"}
Sep 10 22:30:01.326540 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.326489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr" event={"ID":"5f37169d-7606-4139-8a27-8b6b4b671ac0","Type":"ContainerStarted","Data":"fc073ccb067eedfa0e40c79cb50fe68e82c8351605540b8889408808511bf9de"}
Sep 10 22:30:01.326540 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.326537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr" event={"ID":"5f37169d-7606-4139-8a27-8b6b4b671ac0","Type":"ContainerStarted","Data":"2de3ee99fcc602905d8b7f9142b23d6702b045c4f5dd184e0e329929b62f5310"}
Sep 10 22:30:01.338399 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.338358    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2" podStartSLOduration=1.338343709 podStartE2EDuration="1.338343709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:01.337622193 +0000 UTC m=+1470.722127617" watchObservedRunningTime="2026-09-10 22:30:01.338343709 +0000 UTC m=+1470.722849131"
Sep 10 22:30:01.351323 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.351280    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr" podStartSLOduration=1.3512633059999999 podStartE2EDuration="1.351263306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:01.350265493 +0000 UTC m=+1470.734770920" watchObservedRunningTime="2026-09-10 22:30:01.351263306 +0000 UTC m=+1470.735768733"
Sep 10 22:30:01.364766 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:01.364730    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8" podStartSLOduration=1.364720283 podStartE2EDuration="1.364720283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:01.362505008 +0000 UTC m=+1470.747010432" watchObservedRunningTime="2026-09-10 22:30:01.364720283 +0000 UTC m=+1470.749225730"
Sep 10 22:30:02.333226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.333196    2563 generic.go:356] "Generic (PLEG): container finished" podID="e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d" containerID="d8ec5d74a93e34a505c33524c4ada10b3c0e6484faa9ce5c21fe697b7aef769e" exitCode=0
Sep 10 22:30:02.333649 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.333268    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2" event={"ID":"e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d","Type":"ContainerDied","Data":"d8ec5d74a93e34a505c33524c4ada10b3c0e6484faa9ce5c21fe697b7aef769e"}
Sep 10 22:30:02.334785 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.334760    2563 generic.go:356] "Generic (PLEG): container finished" podID="f21aaea6-7184-46eb-a2d3-00050465c831" containerID="aa7979f86811b8e30531914f2a48d2b063173b92424892a6eaa802da8c1913c3" exitCode=0
Sep 10 22:30:02.334884 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.334843    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8" event={"ID":"f21aaea6-7184-46eb-a2d3-00050465c831","Type":"ContainerDied","Data":"aa7979f86811b8e30531914f2a48d2b063173b92424892a6eaa802da8c1913c3"}
Sep 10 22:30:02.336429 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.336406    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f37169d-7606-4139-8a27-8b6b4b671ac0" containerID="fc073ccb067eedfa0e40c79cb50fe68e82c8351605540b8889408808511bf9de" exitCode=0
Sep 10 22:30:02.336549 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.336484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr" event={"ID":"5f37169d-7606-4139-8a27-8b6b4b671ac0","Type":"ContainerDied","Data":"fc073ccb067eedfa0e40c79cb50fe68e82c8351605540b8889408808511bf9de"}
Sep 10 22:30:02.853156 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.853116    2563 patch_prober.go:28] interesting pod/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 22:30:02.853321 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:02.853189    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 22:30:03.538531 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.538504    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"
Sep 10 22:30:03.553071 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.552752    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"
Sep 10 22:30:03.557761 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.557737    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"
Sep 10 22:30:03.684401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.684380    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:30:03.821418 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821390    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-credential-socket\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.821701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821426    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-envoy\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.821701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821506    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-socket\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.821701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821559    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt7g7\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-kube-api-access-kt7g7\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.821701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821646    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-token\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.821701 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821684    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-data\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.821975 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821706    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istiod-ca-cert\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.821975 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821707    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:03.821975 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:03.821975 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.821953    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:03.822187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822029    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-podinfo\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.822187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822040    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:30:03.822187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822070    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-certs\") pod \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\" (UID: \"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7\") "
Sep 10 22:30:03.822330 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:03.822485 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822468    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.822564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822490    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.822564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822504    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.822564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822517    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.822564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.822531    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.823891 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.823859    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:03.824199 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.824183    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:30:03.824458 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.824411    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-kube-api-access-kt7g7" (OuterVolumeSpecName: "kube-api-access-kt7g7") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "kube-api-access-kt7g7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:03.824557 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.824458    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" (UID: "7d34a01c-bf0e-4013-b47c-9b4ab0f695c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:03.923498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.923463    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt7g7\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-kube-api-access-kt7g7\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.923498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.923499    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.923708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.923512    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:03.923708 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:03.923524    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:04.349124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.349089    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"
Sep 10 22:30:04.349319 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.349123    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr" event={"ID":"5f37169d-7606-4139-8a27-8b6b4b671ac0","Type":"ContainerDied","Data":"2de3ee99fcc602905d8b7f9142b23d6702b045c4f5dd184e0e329929b62f5310"}
Sep 10 22:30:04.349319 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.349214    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2de3ee99fcc602905d8b7f9142b23d6702b045c4f5dd184e0e329929b62f5310"
Sep 10 22:30:04.351039 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.351010    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerID="b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2" exitCode=0
Sep 10 22:30:04.351204 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.351086    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"
Sep 10 22:30:04.351204 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.351122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" event={"ID":"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7","Type":"ContainerDied","Data":"b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2"}
Sep 10 22:30:04.351204 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.351199    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59" event={"ID":"7d34a01c-bf0e-4013-b47c-9b4ab0f695c7","Type":"ContainerDied","Data":"d6b9e972366d2a9ea0799254bb13fd08a6071518c10a08091ddcf2c1c8c83cad"}
Sep 10 22:30:04.351405 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.351227    2563 scope.go:117] "RemoveContainer" containerID="b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2"
Sep 10 22:30:04.353316 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.352947    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"
Sep 10 22:30:04.353316 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.352959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2" event={"ID":"e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d","Type":"ContainerDied","Data":"de63c58a3d86b9a5a75550bd72cbb61d8a735c3472a97d279ec742aa5405ffaf"}
Sep 10 22:30:04.353316 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.352997    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de63c58a3d86b9a5a75550bd72cbb61d8a735c3472a97d279ec742aa5405ffaf"
Sep 10 22:30:04.354945 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.354918    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8" event={"ID":"f21aaea6-7184-46eb-a2d3-00050465c831","Type":"ContainerDied","Data":"082bc498f577ac61f992bed987c762ef309ba529c0747b1fded21b519edeb90e"}
Sep 10 22:30:04.354945 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.354944    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"
Sep 10 22:30:04.355154 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.354965    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="082bc498f577ac61f992bed987c762ef309ba529c0747b1fded21b519edeb90e"
Sep 10 22:30:04.367976 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.367959    2563 scope.go:117] "RemoveContainer" containerID="548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1"
Sep 10 22:30:04.377040 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.377020    2563 scope.go:117] "RemoveContainer" containerID="b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2"
Sep 10 22:30:04.377302 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:30:04.377284    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2\": container with ID starting with b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2 not found: ID does not exist" containerID="b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2"
Sep 10 22:30:04.377347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.377310    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2"} err="failed to get container status \"b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2\": rpc error: code = NotFound desc = could not find container \"b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2\": container with ID starting with b3603f2488f9d3c0328def6762b2ac02409654b87cab1e2e578cbe920de254b2 not found: ID does not exist"
Sep 10 22:30:04.377347 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.377328    2563 scope.go:117] "RemoveContainer" containerID="548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1"
Sep 10 22:30:04.377608 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:30:04.377588    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1\": container with ID starting with 548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1 not found: ID does not exist" containerID="548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1"
Sep 10 22:30:04.377679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.377616    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1"} err="failed to get container status \"548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1\": rpc error: code = NotFound desc = could not find container \"548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1\": container with ID starting with 548fd47ef285c1d3671d5e05387221725ab555dd7863edbdbb7f337b51bddba1 not found: ID does not exist"
Sep 10 22:30:04.381750 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.381731    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"]
Sep 10 22:30:04.390722 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:04.390702    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-19922b-openshift-default-7644d4fd66-fwc59"]
Sep 10 22:30:05.110888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:05.110857    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" path="/var/lib/kubelet/pods/7d34a01c-bf0e-4013-b47c-9b4ab0f695c7/volumes"
Sep 10 22:30:14.234947 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.234916    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"]
Sep 10 22:30:14.235386 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.235118    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8" podUID="43b75795-5d05-45dc-95b5-2b97c5c78657" containerName="payload-processing" containerID="cri-o://c2f20f8e839042c558c6fe85e5c50bc1296ad1716bc3cd9a3f98d3297bbc7628" gracePeriod=30
Sep 10 22:30:14.410304 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.410275    2563 generic.go:356] "Generic (PLEG): container finished" podID="43b75795-5d05-45dc-95b5-2b97c5c78657" containerID="c2f20f8e839042c558c6fe85e5c50bc1296ad1716bc3cd9a3f98d3297bbc7628" exitCode=0
Sep 10 22:30:14.410460 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.410340    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8" event={"ID":"43b75795-5d05-45dc-95b5-2b97c5c78657","Type":"ContainerDied","Data":"c2f20f8e839042c558c6fe85e5c50bc1296ad1716bc3cd9a3f98d3297bbc7628"}
Sep 10 22:30:14.497952 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.497888    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:30:14.523908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.523877    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43b75795-5d05-45dc-95b5-2b97c5c78657-plugins-config-volume\") pod \"43b75795-5d05-45dc-95b5-2b97c5c78657\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") "
Sep 10 22:30:14.524063 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.523967    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtxt7\" (UniqueName: \"kubernetes.io/projected/43b75795-5d05-45dc-95b5-2b97c5c78657-kube-api-access-mtxt7\") pod \"43b75795-5d05-45dc-95b5-2b97c5c78657\" (UID: \"43b75795-5d05-45dc-95b5-2b97c5c78657\") "
Sep 10 22:30:14.524205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.524185    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43b75795-5d05-45dc-95b5-2b97c5c78657-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "43b75795-5d05-45dc-95b5-2b97c5c78657" (UID: "43b75795-5d05-45dc-95b5-2b97c5c78657"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:30:14.524273 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.524255    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43b75795-5d05-45dc-95b5-2b97c5c78657-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:14.526199 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.526179    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b75795-5d05-45dc-95b5-2b97c5c78657-kube-api-access-mtxt7" (OuterVolumeSpecName: "kube-api-access-mtxt7") pod "43b75795-5d05-45dc-95b5-2b97c5c78657" (UID: "43b75795-5d05-45dc-95b5-2b97c5c78657"). InnerVolumeSpecName "kube-api-access-mtxt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:14.625150 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:14.625110    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtxt7\" (UniqueName: \"kubernetes.io/projected/43b75795-5d05-45dc-95b5-2b97c5c78657-kube-api-access-mtxt7\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:15.417229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:15.417139    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8" event={"ID":"43b75795-5d05-45dc-95b5-2b97c5c78657","Type":"ContainerDied","Data":"22e1a5128302da0081b8836e7755593bfc26e83f3ece547de21750454317c924"}
Sep 10 22:30:15.417229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:15.417186    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"
Sep 10 22:30:15.417229 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:15.417227    2563 scope.go:117] "RemoveContainer" containerID="c2f20f8e839042c558c6fe85e5c50bc1296ad1716bc3cd9a3f98d3297bbc7628"
Sep 10 22:30:15.433336 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:15.433301    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"]
Sep 10 22:30:15.435221 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:15.435202    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-ad8659-79c4bb7bff-qxzx8"]
Sep 10 22:30:17.110642 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:17.110599    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43b75795-5d05-45dc-95b5-2b97c5c78657" path="/var/lib/kubelet/pods/43b75795-5d05-45dc-95b5-2b97c5c78657/volumes"
Sep 10 22:30:19.102261 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:19.102217    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:30:20.784113 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:20.784079    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"]
Sep 10 22:30:20.784636 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:20.784297    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy" containerID="cri-o://59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781" gracePeriod=30
Sep 10 22:30:21.216664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.216586    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"]
Sep 10 22:30:21.216964 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.216952    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d" containerName="cleanup"
Sep 10 22:30:21.217008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.216966    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d" containerName="cleanup"
Sep 10 22:30:21.217008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.216976    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f21aaea6-7184-46eb-a2d3-00050465c831" containerName="cleanup"
Sep 10 22:30:21.217008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.216981    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21aaea6-7184-46eb-a2d3-00050465c831" containerName="cleanup"
Sep 10 22:30:21.217008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217002    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy"
Sep 10 22:30:21.217008 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217008    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217013    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f37169d-7606-4139-8a27-8b6b4b671ac0" containerName="cleanup"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217018    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f37169d-7606-4139-8a27-8b6b4b671ac0" containerName="cleanup"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217029    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43b75795-5d05-45dc-95b5-2b97c5c78657" containerName="payload-processing"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217034    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="43b75795-5d05-45dc-95b5-2b97c5c78657" containerName="payload-processing"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217052    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217057    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217119    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="43b75795-5d05-45dc-95b5-2b97c5c78657" containerName="payload-processing"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217126    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217132    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f37169d-7606-4139-8a27-8b6b4b671ac0" containerName="cleanup"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217138    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d" containerName="cleanup"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217145    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d34a01c-bf0e-4013-b47c-9b4ab0f695c7" containerName="istio-proxy"
Sep 10 22:30:21.217165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.217153    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f21aaea6-7184-46eb-a2d3-00050465c831" containerName="cleanup"
Sep 10 22:30:21.221543 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.221525    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.223938 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.223916    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"kube-root-ca.crt\""
Sep 10 22:30:21.224095 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.223924    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"openshift-service-ca.crt\""
Sep 10 22:30:21.224095 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.223998    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"default-dockercfg-567d7\""
Sep 10 22:30:21.224206 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.224160    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-0ada08\"/\"rate-test-model-0ada08-kserve-self-signed-certs\""
Sep 10 22:30:21.231362 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.231336    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"]
Sep 10 22:30:21.280277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.280242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2rmj\" (UniqueName: \"kubernetes.io/projected/691828cc-e296-4976-be41-21690d0a0e6e-kube-api-access-s2rmj\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.280471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.280282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-model-cache\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.280471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.280309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-dshm\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.280471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.280416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-home\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.280591 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.280495    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-tmp-dir\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.280591 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.280520    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/691828cc-e296-4976-be41-21690d0a0e6e-tls-certs\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381337 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-home\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381337 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-tmp-dir\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/691828cc-e296-4976-be41-21690d0a0e6e-tls-certs\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2rmj\" (UniqueName: \"kubernetes.io/projected/691828cc-e296-4976-be41-21690d0a0e6e-kube-api-access-s2rmj\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-model-cache\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381607 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-dshm\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381820 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381794    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-home\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381820 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381812    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-tmp-dir\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.381898 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.381885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-model-cache\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.383892 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.383869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-dshm\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.384168 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.384151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/691828cc-e296-4976-be41-21690d0a0e6e-tls-certs\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.389766 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.389745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2rmj\" (UniqueName: \"kubernetes.io/projected/691828cc-e296-4976-be41-21690d0a0e6e-kube-api-access-s2rmj\") pod \"rate-test-model-0ada08-kserve-58d69fcf75-vkvcs\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") " pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.531598 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.531563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:21.804751 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:21.804728    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"]
Sep 10 22:30:21.806855 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:30:21.806825    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod691828cc_e296_4976_be41_21690d0a0e6e.slice/crio-e2e23990bf5a32718eeecd8273e9e073bd796febe7134af7cc496c3a8559f241 WatchSource:0}: Error finding container e2e23990bf5a32718eeecd8273e9e073bd796febe7134af7cc496c3a8559f241: Status 404 returned error can't find the container with id e2e23990bf5a32718eeecd8273e9e073bd796febe7134af7cc496c3a8559f241
Sep 10 22:30:22.455318 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:22.455284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs" event={"ID":"691828cc-e296-4976-be41-21690d0a0e6e","Type":"ContainerStarted","Data":"9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba"}
Sep 10 22:30:22.455503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:22.455325    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs" event={"ID":"691828cc-e296-4976-be41-21690d0a0e6e","Type":"ContainerStarted","Data":"e2e23990bf5a32718eeecd8273e9e073bd796febe7134af7cc496c3a8559f241"}
Sep 10 22:30:22.473801 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:22.473745    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs" podStartSLOduration=1.473727679 podStartE2EDuration="1.473727679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:30:22.471935463 +0000 UTC m=+1491.856440890" watchObservedRunningTime="2026-09-10 22:30:22.473727679 +0000 UTC m=+1491.858233103"
Sep 10 22:30:24.802984 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:24.802949    2563 patch_prober.go:28] interesting pod/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 22:30:24.803349 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:24.803018    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 22:30:26.045700 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.045643    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:30:26.124712 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-credential-socket\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.124886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124761    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-certs\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.124886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124797    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:26.124886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124807    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-data\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.124886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrx7q\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-kube-api-access-hrx7q\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.124886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-token\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.125157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124924    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-podinfo\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.125157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.124991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7634911c-2073-4b6d-a8e0-af7fe7167434-istiod-ca-cert\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.125157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125025    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-socket\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.125157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125037    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:26.125157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125053    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-envoy\") pod \"7634911c-2073-4b6d-a8e0-af7fe7167434\" (UID: \"7634911c-2073-4b6d-a8e0-af7fe7167434\") "
Sep 10 22:30:26.125157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125054    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-data" (OuterVolumeSpecName: "istio-data") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:26.125471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125276    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:26.125471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125390    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.125471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125393    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7634911c-2073-4b6d-a8e0-af7fe7167434-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:30:26.125471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125406    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.125471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125420    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.125471 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.125432    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.127210 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.127166    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-token" (OuterVolumeSpecName: "istio-token") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:26.127210 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.127193    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:30:26.127352 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.127253    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-kube-api-access-hrx7q" (OuterVolumeSpecName: "kube-api-access-hrx7q") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "kube-api-access-hrx7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:26.127352 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.127264    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7634911c-2073-4b6d-a8e0-af7fe7167434" (UID: "7634911c-2073-4b6d-a8e0-af7fe7167434"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:30:26.226811 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.226771    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.226811 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.226802    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.226811 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.226812    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7634911c-2073-4b6d-a8e0-af7fe7167434-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.226811 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.226820    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7634911c-2073-4b6d-a8e0-af7fe7167434-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.227065 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.226829    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrx7q\" (UniqueName: \"kubernetes.io/projected/7634911c-2073-4b6d-a8e0-af7fe7167434-kube-api-access-hrx7q\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:26.478508 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.478403    2563 generic.go:356] "Generic (PLEG): container finished" podID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerID="59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781" exitCode=0
Sep 10 22:30:26.478508 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.478472    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"
Sep 10 22:30:26.478508 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.478491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" event={"ID":"7634911c-2073-4b6d-a8e0-af7fe7167434","Type":"ContainerDied","Data":"59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781"}
Sep 10 22:30:26.478802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.478543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn" event={"ID":"7634911c-2073-4b6d-a8e0-af7fe7167434","Type":"ContainerDied","Data":"8438ced260f79c9a9f59fd71dc5bc33f6baacff40a85fbc243d1ee7c7a5be996"}
Sep 10 22:30:26.478802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.478568    2563 scope.go:117] "RemoveContainer" containerID="59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781"
Sep 10 22:30:26.488257 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.488242    2563 scope.go:117] "RemoveContainer" containerID="2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e"
Sep 10 22:30:26.496461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.496424    2563 scope.go:117] "RemoveContainer" containerID="59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781"
Sep 10 22:30:26.496709 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:30:26.496692    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781\": container with ID starting with 59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781 not found: ID does not exist" containerID="59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781"
Sep 10 22:30:26.496752 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.496717    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781"} err="failed to get container status \"59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781\": rpc error: code = NotFound desc = could not find container \"59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781\": container with ID starting with 59bb4ecd1242648f946e5cad1874c3a1df4b8afcfe709188f364d174c3529781 not found: ID does not exist"
Sep 10 22:30:26.496752 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.496733    2563 scope.go:117] "RemoveContainer" containerID="2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e"
Sep 10 22:30:26.497026 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:30:26.497003    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e\": container with ID starting with 2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e not found: ID does not exist" containerID="2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e"
Sep 10 22:30:26.497106 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.497036    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e"} err="failed to get container status \"2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e\": rpc error: code = NotFound desc = could not find container \"2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e\": container with ID starting with 2597fc7b43fdc689c9bfdb37658105d5fe62aabc72bffe137fdb3cb14abd4f3e not found: ID does not exist"
Sep 10 22:30:26.499779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.499758    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"]
Sep 10 22:30:26.505914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:26.505894    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-ad8659-openshift-default-58df4b4b8d-x4zpn"]
Sep 10 22:30:27.111342 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:27.111308    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" path="/var/lib/kubelet/pods/7634911c-2073-4b6d-a8e0-af7fe7167434/volumes"
Sep 10 22:30:31.339990 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.339956    2563 scope.go:117] "RemoveContainer" containerID="4aa38bd61edae8f64884bed0de50a72294d5402120c481a82a51b7b3ffa4b1ba"
Sep 10 22:30:31.532451 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.532403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:31.532697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.532681    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:31.546858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.546833    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:31.557574 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.557405    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"]
Sep 10 22:30:31.559407 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.559385    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-4a864c-29817990-vwvr2"]
Sep 10 22:30:31.593637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.593547    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"]
Sep 10 22:30:31.593831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.593798    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p" podUID="17626a83-4c8a-4678-a3e9-4c78d16126f5" containerName="payload-pre-processing" containerID="cri-o://9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c" gracePeriod=30
Sep 10 22:30:31.853695 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.853631    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:30:31.977935 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.977904    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6qxb\" (UniqueName: \"kubernetes.io/projected/17626a83-4c8a-4678-a3e9-4c78d16126f5-kube-api-access-j6qxb\") pod \"17626a83-4c8a-4678-a3e9-4c78d16126f5\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") "
Sep 10 22:30:31.978143 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.977999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17626a83-4c8a-4678-a3e9-4c78d16126f5-plugins-config-volume\") pod \"17626a83-4c8a-4678-a3e9-4c78d16126f5\" (UID: \"17626a83-4c8a-4678-a3e9-4c78d16126f5\") "
Sep 10 22:30:31.978399 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.978374    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17626a83-4c8a-4678-a3e9-4c78d16126f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "17626a83-4c8a-4678-a3e9-4c78d16126f5" (UID: "17626a83-4c8a-4678-a3e9-4c78d16126f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:30:31.980179 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:31.980162    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17626a83-4c8a-4678-a3e9-4c78d16126f5-kube-api-access-j6qxb" (OuterVolumeSpecName: "kube-api-access-j6qxb") pod "17626a83-4c8a-4678-a3e9-4c78d16126f5" (UID: "17626a83-4c8a-4678-a3e9-4c78d16126f5"). InnerVolumeSpecName "kube-api-access-j6qxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:32.078888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.078852    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17626a83-4c8a-4678-a3e9-4c78d16126f5-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:32.078888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.078883    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6qxb\" (UniqueName: \"kubernetes.io/projected/17626a83-4c8a-4678-a3e9-4c78d16126f5-kube-api-access-j6qxb\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:32.514009 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.513977    2563 generic.go:356] "Generic (PLEG): container finished" podID="17626a83-4c8a-4678-a3e9-4c78d16126f5" containerID="9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c" exitCode=0
Sep 10 22:30:32.514723 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.514022    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"
Sep 10 22:30:32.514723 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.514059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p" event={"ID":"17626a83-4c8a-4678-a3e9-4c78d16126f5","Type":"ContainerDied","Data":"9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c"}
Sep 10 22:30:32.514723 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.514128    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p" event={"ID":"17626a83-4c8a-4678-a3e9-4c78d16126f5","Type":"ContainerDied","Data":"4d6a68ab8877e56f9de6782def86b4e90b54215854c34f01ea5820430cc5e2fd"}
Sep 10 22:30:32.514723 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.514163    2563 scope.go:117] "RemoveContainer" containerID="9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c"
Sep 10 22:30:32.524115 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.524097    2563 scope.go:117] "RemoveContainer" containerID="9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c"
Sep 10 22:30:32.524413 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:30:32.524394    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c\": container with ID starting with 9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c not found: ID does not exist" containerID="9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c"
Sep 10 22:30:32.524513 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.524424    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c"} err="failed to get container status \"9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c\": rpc error: code = NotFound desc = could not find container \"9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c\": container with ID starting with 9e59f9b858bdcdab3278e63bc655ab2743ec35a0eb5e6f8b6052101ac9fd109c not found: ID does not exist"
Sep 10 22:30:32.527015 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.526997    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:30:32.534344 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.534323    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"]
Sep 10 22:30:32.537987 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:32.537965    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4a864c-6cd4cf586b-9kr5p"]
Sep 10 22:30:33.111850 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:33.111819    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17626a83-4c8a-4678-a3e9-4c78d16126f5" path="/var/lib/kubelet/pods/17626a83-4c8a-4678-a3e9-4c78d16126f5/volumes"
Sep 10 22:30:33.112101 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:33.112087    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d" path="/var/lib/kubelet/pods/e259aded-5c9c-4fcc-85fa-4dfe9f8fad4d/volumes"
Sep 10 22:30:35.304866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:35.304831    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:30:48.958177 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:48.956942    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"]
Sep 10 22:30:48.958177 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:48.957208    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l" podUID="c7bbc96c-4571-416b-aa33-ab6883775aa4" containerName="payload-processing" containerID="cri-o://eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7" gracePeriod=30
Sep 10 22:30:48.976535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:48.976509    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"]
Sep 10 22:30:48.976972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:48.976916    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8" podUID="a272dd6c-e87d-4acf-9ca5-ffa1023f7209" containerName="payload-pre-processing" containerID="cri-o://8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138" gracePeriod=30
Sep 10 22:30:49.253328 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.253305    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:30:49.255977 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.255959    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:30:49.336586 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.336556    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-plugins-config-volume\") pod \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") "
Sep 10 22:30:49.336737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.336612    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7bbc96c-4571-416b-aa33-ab6883775aa4-plugins-config-volume\") pod \"c7bbc96c-4571-416b-aa33-ab6883775aa4\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") "
Sep 10 22:30:49.336737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.336672    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g56s\" (UniqueName: \"kubernetes.io/projected/c7bbc96c-4571-416b-aa33-ab6883775aa4-kube-api-access-9g56s\") pod \"c7bbc96c-4571-416b-aa33-ab6883775aa4\" (UID: \"c7bbc96c-4571-416b-aa33-ab6883775aa4\") "
Sep 10 22:30:49.336737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.336728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmt4h\" (UniqueName: \"kubernetes.io/projected/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-kube-api-access-xmt4h\") pod \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\" (UID: \"a272dd6c-e87d-4acf-9ca5-ffa1023f7209\") "
Sep 10 22:30:49.337021 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.336996    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a272dd6c-e87d-4acf-9ca5-ffa1023f7209" (UID: "a272dd6c-e87d-4acf-9ca5-ffa1023f7209"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:30:49.337070 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.336996    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7bbc96c-4571-416b-aa33-ab6883775aa4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7bbc96c-4571-416b-aa33-ab6883775aa4" (UID: "c7bbc96c-4571-416b-aa33-ab6883775aa4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:30:49.339026 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.339003    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7bbc96c-4571-416b-aa33-ab6883775aa4-kube-api-access-9g56s" (OuterVolumeSpecName: "kube-api-access-9g56s") pod "c7bbc96c-4571-416b-aa33-ab6883775aa4" (UID: "c7bbc96c-4571-416b-aa33-ab6883775aa4"). InnerVolumeSpecName "kube-api-access-9g56s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:49.339131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.339113    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-kube-api-access-xmt4h" (OuterVolumeSpecName: "kube-api-access-xmt4h") pod "a272dd6c-e87d-4acf-9ca5-ffa1023f7209" (UID: "a272dd6c-e87d-4acf-9ca5-ffa1023f7209"). InnerVolumeSpecName "kube-api-access-xmt4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:30:49.437644 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.437617    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7bbc96c-4571-416b-aa33-ab6883775aa4-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:49.437644 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.437641    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g56s\" (UniqueName: \"kubernetes.io/projected/c7bbc96c-4571-416b-aa33-ab6883775aa4-kube-api-access-9g56s\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:49.437644 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.437650    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmt4h\" (UniqueName: \"kubernetes.io/projected/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-kube-api-access-xmt4h\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:49.437840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.437659    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a272dd6c-e87d-4acf-9ca5-ffa1023f7209-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:30:49.606045 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.606004    2563 generic.go:356] "Generic (PLEG): container finished" podID="a272dd6c-e87d-4acf-9ca5-ffa1023f7209" containerID="8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138" exitCode=0
Sep 10 22:30:49.606236 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.606056    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"
Sep 10 22:30:49.606236 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.606090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8" event={"ID":"a272dd6c-e87d-4acf-9ca5-ffa1023f7209","Type":"ContainerDied","Data":"8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138"}
Sep 10 22:30:49.606236 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.606151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8" event={"ID":"a272dd6c-e87d-4acf-9ca5-ffa1023f7209","Type":"ContainerDied","Data":"197c33fa98b4e3dbe8fe6981ccc67c17eeeea8f9f6e29bb8784be17c98f1e5f0"}
Sep 10 22:30:49.606236 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.606175    2563 scope.go:117] "RemoveContainer" containerID="8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138"
Sep 10 22:30:49.607584 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.607561    2563 generic.go:356] "Generic (PLEG): container finished" podID="c7bbc96c-4571-416b-aa33-ab6883775aa4" containerID="eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7" exitCode=0
Sep 10 22:30:49.607812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.607598    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"
Sep 10 22:30:49.607812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.607640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l" event={"ID":"c7bbc96c-4571-416b-aa33-ab6883775aa4","Type":"ContainerDied","Data":"eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7"}
Sep 10 22:30:49.607812 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.607692    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l" event={"ID":"c7bbc96c-4571-416b-aa33-ab6883775aa4","Type":"ContainerDied","Data":"621ea7494462b8601409fa755d82bbe65b1ec3479eb5445cd855034e6dacef29"}
Sep 10 22:30:49.615673 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.615651    2563 scope.go:117] "RemoveContainer" containerID="8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138"
Sep 10 22:30:49.615975 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:30:49.615955    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138\": container with ID starting with 8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138 not found: ID does not exist" containerID="8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138"
Sep 10 22:30:49.616068 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.615982    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138"} err="failed to get container status \"8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138\": rpc error: code = NotFound desc = could not find container \"8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138\": container with ID starting with 8e39e5c8a334a55b552f9981f2180f66aae6c8f854b29f340b0fabd37cfc7138 not found: ID does not exist"
Sep 10 22:30:49.616068 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.615998    2563 scope.go:117] "RemoveContainer" containerID="eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7"
Sep 10 22:30:49.624272 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.624255    2563 scope.go:117] "RemoveContainer" containerID="eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7"
Sep 10 22:30:49.624550 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:30:49.624535    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7\": container with ID starting with eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7 not found: ID does not exist" containerID="eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7"
Sep 10 22:30:49.624600 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.624557    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7"} err="failed to get container status \"eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7\": rpc error: code = NotFound desc = could not find container \"eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7\": container with ID starting with eff7f39edbe8a37df8701554777c3f52c77df0d8693eb9236d5e9018f2ce5de7 not found: ID does not exist"
Sep 10 22:30:49.652852 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.652832    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"]
Sep 10 22:30:49.656355 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.656336    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-32343f-68d7bbd594-9rcd8"]
Sep 10 22:30:49.681224 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.681201    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"]
Sep 10 22:30:49.687213 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:49.687194    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-32343f-6fddb478c7-wjj2l"]
Sep 10 22:30:51.111268 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:51.111235    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a272dd6c-e87d-4acf-9ca5-ffa1023f7209" path="/var/lib/kubelet/pods/a272dd6c-e87d-4acf-9ca5-ffa1023f7209/volumes"
Sep 10 22:30:51.111649 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:51.111637    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7bbc96c-4571-416b-aa33-ab6883775aa4" path="/var/lib/kubelet/pods/c7bbc96c-4571-416b-aa33-ab6883775aa4/volumes"
Sep 10 22:30:54.702886 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:30:54.702806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:31:05.903063 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:05.903026    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:31:12.095723 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:12.095689    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"]
Sep 10 22:31:12.736016 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:12.735978    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"]
Sep 10 22:31:16.000922 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:16.000885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:31:26.094627 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:26.094595    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:31:27.115456 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.115402    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74" podUID="403d9185-479c-4070-94e1-9860cac88819" containerName="main" containerID="cri-o://1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b" gracePeriod=45
Sep 10 22:31:27.365164 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.365134    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:31:27.478406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478329    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403d9185-479c-4070-94e1-9860cac88819-tls-certs\") pod \"403d9185-479c-4070-94e1-9860cac88819\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") "
Sep 10 22:31:27.478406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478384    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-tmp-dir\") pod \"403d9185-479c-4070-94e1-9860cac88819\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") "
Sep 10 22:31:27.478633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2zbs\" (UniqueName: \"kubernetes.io/projected/403d9185-479c-4070-94e1-9860cac88819-kube-api-access-n2zbs\") pod \"403d9185-479c-4070-94e1-9860cac88819\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") "
Sep 10 22:31:27.478633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478474    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-dshm\") pod \"403d9185-479c-4070-94e1-9860cac88819\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") "
Sep 10 22:31:27.478633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478551    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-home\") pod \"403d9185-479c-4070-94e1-9860cac88819\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") "
Sep 10 22:31:27.478633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478624    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-model-cache\") pod \"403d9185-479c-4070-94e1-9860cac88819\" (UID: \"403d9185-479c-4070-94e1-9860cac88819\") "
Sep 10 22:31:27.478859 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478684    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "403d9185-479c-4070-94e1-9860cac88819" (UID: "403d9185-479c-4070-94e1-9860cac88819"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:27.478859 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-model-cache" (OuterVolumeSpecName: "model-cache") pod "403d9185-479c-4070-94e1-9860cac88819" (UID: "403d9185-479c-4070-94e1-9860cac88819"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:27.478946 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478887    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-home" (OuterVolumeSpecName: "home") pod "403d9185-479c-4070-94e1-9860cac88819" (UID: "403d9185-479c-4070-94e1-9860cac88819"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:27.478996 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478949    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:27.478996 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.478969    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:27.480731 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.480706    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/403d9185-479c-4070-94e1-9860cac88819-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "403d9185-479c-4070-94e1-9860cac88819" (UID: "403d9185-479c-4070-94e1-9860cac88819"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:31:27.480849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.480747    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-dshm" (OuterVolumeSpecName: "dshm") pod "403d9185-479c-4070-94e1-9860cac88819" (UID: "403d9185-479c-4070-94e1-9860cac88819"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:27.480849 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.480762    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403d9185-479c-4070-94e1-9860cac88819-kube-api-access-n2zbs" (OuterVolumeSpecName: "kube-api-access-n2zbs") pod "403d9185-479c-4070-94e1-9860cac88819" (UID: "403d9185-479c-4070-94e1-9860cac88819"). InnerVolumeSpecName "kube-api-access-n2zbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:31:27.580028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.579995    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:27.580028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.580022    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/403d9185-479c-4070-94e1-9860cac88819-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:27.580200 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.580038    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2zbs\" (UniqueName: \"kubernetes.io/projected/403d9185-479c-4070-94e1-9860cac88819-kube-api-access-n2zbs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:27.580200 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.580049    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/403d9185-479c-4070-94e1-9860cac88819-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:27.756096 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.756031    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs" podUID="691828cc-e296-4976-be41-21690d0a0e6e" containerName="main" containerID="cri-o://9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba" gracePeriod=45
Sep 10 22:31:27.810346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.810318    2563 generic.go:356] "Generic (PLEG): container finished" podID="403d9185-479c-4070-94e1-9860cac88819" containerID="1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b" exitCode=0
Sep 10 22:31:27.810520 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.810371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74" event={"ID":"403d9185-479c-4070-94e1-9860cac88819","Type":"ContainerDied","Data":"1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b"}
Sep 10 22:31:27.810520 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.810411    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74" event={"ID":"403d9185-479c-4070-94e1-9860cac88819","Type":"ContainerDied","Data":"3a51dcee08ff56ee1b02d41d13dba18be268c0cd22542bfb642ce0d22b54d67e"}
Sep 10 22:31:27.810520 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.810425    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"
Sep 10 22:31:27.810661 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.810430    2563 scope.go:117] "RemoveContainer" containerID="1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b"
Sep 10 22:31:27.829997 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.829969    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"]
Sep 10 22:31:27.832108 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.832086    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-e367c1/rate-test-model-e367c1-kserve-86d8b6977f-zqn74"]
Sep 10 22:31:27.867308 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.867282    2563 scope.go:117] "RemoveContainer" containerID="1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b"
Sep 10 22:31:27.867674 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:31:27.867651    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b\": container with ID starting with 1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b not found: ID does not exist" containerID="1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b"
Sep 10 22:31:27.867798 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.867682    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b"} err="failed to get container status \"1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b\": rpc error: code = NotFound desc = could not find container \"1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b\": container with ID starting with 1a8b9b385dee87a035605781bffdf76d68a06cb71b1f82417fc1a8d48fb9e31b not found: ID does not exist"
Sep 10 22:31:27.993156 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:27.993131    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:31:28.084092 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084023    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-model-cache\") pod \"691828cc-e296-4976-be41-21690d0a0e6e\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") "
Sep 10 22:31:28.084092 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-tmp-dir\") pod \"691828cc-e296-4976-be41-21690d0a0e6e\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") "
Sep 10 22:31:28.084249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084096    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-dshm\") pod \"691828cc-e296-4976-be41-21690d0a0e6e\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") "
Sep 10 22:31:28.084249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084139    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2rmj\" (UniqueName: \"kubernetes.io/projected/691828cc-e296-4976-be41-21690d0a0e6e-kube-api-access-s2rmj\") pod \"691828cc-e296-4976-be41-21690d0a0e6e\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") "
Sep 10 22:31:28.084249 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084220    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-home\") pod \"691828cc-e296-4976-be41-21690d0a0e6e\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") "
Sep 10 22:31:28.084370 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/691828cc-e296-4976-be41-21690d0a0e6e-tls-certs\") pod \"691828cc-e296-4976-be41-21690d0a0e6e\" (UID: \"691828cc-e296-4976-be41-21690d0a0e6e\") "
Sep 10 22:31:28.084370 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084308    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "691828cc-e296-4976-be41-21690d0a0e6e" (UID: "691828cc-e296-4976-be41-21690d0a0e6e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:28.084370 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084327    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-model-cache" (OuterVolumeSpecName: "model-cache") pod "691828cc-e296-4976-be41-21690d0a0e6e" (UID: "691828cc-e296-4976-be41-21690d0a0e6e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:28.084551 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084525    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-home" (OuterVolumeSpecName: "home") pod "691828cc-e296-4976-be41-21690d0a0e6e" (UID: "691828cc-e296-4976-be41-21690d0a0e6e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:28.084551 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084544    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:28.084702 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.084569    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:28.086384 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.086362    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-dshm" (OuterVolumeSpecName: "dshm") pod "691828cc-e296-4976-be41-21690d0a0e6e" (UID: "691828cc-e296-4976-be41-21690d0a0e6e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:31:28.086477 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.086365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/691828cc-e296-4976-be41-21690d0a0e6e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "691828cc-e296-4976-be41-21690d0a0e6e" (UID: "691828cc-e296-4976-be41-21690d0a0e6e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:31:28.086477 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.086423    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/691828cc-e296-4976-be41-21690d0a0e6e-kube-api-access-s2rmj" (OuterVolumeSpecName: "kube-api-access-s2rmj") pod "691828cc-e296-4976-be41-21690d0a0e6e" (UID: "691828cc-e296-4976-be41-21690d0a0e6e"). InnerVolumeSpecName "kube-api-access-s2rmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:31:28.185404 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.185371    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:28.185404 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.185400    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/691828cc-e296-4976-be41-21690d0a0e6e-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:28.185404 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.185410    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/691828cc-e296-4976-be41-21690d0a0e6e-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:28.185855 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.185419    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2rmj\" (UniqueName: \"kubernetes.io/projected/691828cc-e296-4976-be41-21690d0a0e6e-kube-api-access-s2rmj\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:31:28.817714 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.817680    2563 generic.go:356] "Generic (PLEG): container finished" podID="691828cc-e296-4976-be41-21690d0a0e6e" containerID="9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba" exitCode=0
Sep 10 22:31:28.817908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.817726    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"
Sep 10 22:31:28.817908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.817769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs" event={"ID":"691828cc-e296-4976-be41-21690d0a0e6e","Type":"ContainerDied","Data":"9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba"}
Sep 10 22:31:28.817908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.817813    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs" event={"ID":"691828cc-e296-4976-be41-21690d0a0e6e","Type":"ContainerDied","Data":"e2e23990bf5a32718eeecd8273e9e073bd796febe7134af7cc496c3a8559f241"}
Sep 10 22:31:28.817908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.817834    2563 scope.go:117] "RemoveContainer" containerID="9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba"
Sep 10 22:31:28.826761 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.826745    2563 scope.go:117] "RemoveContainer" containerID="9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba"
Sep 10 22:31:28.827033 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:31:28.827016    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba\": container with ID starting with 9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba not found: ID does not exist" containerID="9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba"
Sep 10 22:31:28.827072 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.827045    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba"} err="failed to get container status \"9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba\": rpc error: code = NotFound desc = could not find container \"9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba\": container with ID starting with 9713e88969a37fbdb44aecae44ac628b7130f0f748b8a2b1240e0e8b8ab39fba not found: ID does not exist"
Sep 10 22:31:28.842728 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.840467    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"]
Sep 10 22:31:28.846069 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:28.846044    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-0ada08/rate-test-model-0ada08-kserve-58d69fcf75-vkvcs"]
Sep 10 22:31:29.111083 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:29.111010    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="403d9185-479c-4070-94e1-9860cac88819" path="/var/lib/kubelet/pods/403d9185-479c-4070-94e1-9860cac88819/volumes"
Sep 10 22:31:29.111370 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:29.111357    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="691828cc-e296-4976-be41-21690d0a0e6e" path="/var/lib/kubelet/pods/691828cc-e296-4976-be41-21690d0a0e6e/volumes"
Sep 10 22:31:42.115816 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.115783    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"]
Sep 10 22:31:42.116326 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116307    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116329    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116345    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403d9185-479c-4070-94e1-9860cac88819" containerName="main"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116353    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="403d9185-479c-4070-94e1-9860cac88819" containerName="main"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116365    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17626a83-4c8a-4678-a3e9-4c78d16126f5" containerName="payload-pre-processing"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116373    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17626a83-4c8a-4678-a3e9-4c78d16126f5" containerName="payload-pre-processing"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116385    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116393    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy"
Sep 10 22:31:42.116409 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116409    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a272dd6c-e87d-4acf-9ca5-ffa1023f7209" containerName="payload-pre-processing"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116416    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a272dd6c-e87d-4acf-9ca5-ffa1023f7209" containerName="payload-pre-processing"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116426    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7bbc96c-4571-416b-aa33-ab6883775aa4" containerName="payload-processing"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116452    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7bbc96c-4571-416b-aa33-ab6883775aa4" containerName="payload-processing"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116474    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="691828cc-e296-4976-be41-21690d0a0e6e" containerName="main"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116482    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="691828cc-e296-4976-be41-21690d0a0e6e" containerName="main"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116585    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116597    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17626a83-4c8a-4678-a3e9-4c78d16126f5" containerName="payload-pre-processing"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116606    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7bbc96c-4571-416b-aa33-ab6883775aa4" containerName="payload-processing"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116619    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a272dd6c-e87d-4acf-9ca5-ffa1023f7209" containerName="payload-pre-processing"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116629    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="691828cc-e296-4976-be41-21690d0a0e6e" containerName="main"
Sep 10 22:31:42.116821 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.116637    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="403d9185-479c-4070-94e1-9860cac88819" containerName="main"
Sep 10 22:31:42.121095 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.121075    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.123838 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.123821    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-9b2092-openshift-default-dockercfg-kdt8d\""
Sep 10 22:31:42.140461 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.140412    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"]
Sep 10 22:31:42.209251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-credential-socket\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-certs\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209478 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-envoy\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209478 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c15550-c70e-4fce-a797-3cef528070db-istio-podinfo\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209478 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96nlq\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-kube-api-access-96nlq\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209478 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-istio-token\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-socket\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c15550-c70e-4fce-a797-3cef528070db-istiod-ca-cert\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.209624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.209537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-data\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.310391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-envoy\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.310391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c15550-c70e-4fce-a797-3cef528070db-istio-podinfo\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.310633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96nlq\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-kube-api-access-96nlq\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.310633 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-istio-token\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.310736 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-socket\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.310788 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c15550-c70e-4fce-a797-3cef528070db-istiod-ca-cert\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.310923 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-data\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.311001 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-credential-socket\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.311001 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.310996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-certs\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.311140 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.311108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-socket\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.311433 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.311345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-credential-socket\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.311433 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.311350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-certs\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.311599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.311478    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-data\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.311599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.311486    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c15550-c70e-4fce-a797-3cef528070db-istiod-ca-cert\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.312959 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.312939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-envoy\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.313179 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.313160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c15550-c70e-4fce-a797-3cef528070db-istio-podinfo\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.317927 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.317908    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96nlq\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-kube-api-access-96nlq\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.318028 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.317993    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-istio-token\") pod \"e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") " pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.430889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.430815    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:42.704493 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.704469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"]
Sep 10 22:31:42.706598 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:31:42.706573    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c15550_c70e_4fce_a797_3cef528070db.slice/crio-3e7ab5c6b013c817dade40b3f85e9b8edd80d6f7c337c74f44dc4df2f28e012d WatchSource:0}: Error finding container 3e7ab5c6b013c817dade40b3f85e9b8edd80d6f7c337c74f44dc4df2f28e012d: Status 404 returned error can't find the container with id 3e7ab5c6b013c817dade40b3f85e9b8edd80d6f7c337c74f44dc4df2f28e012d
Sep 10 22:31:42.709041 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.709013    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:31:42.709113 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.709100    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:31:42.709155 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.709131    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:31:42.895457 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.895394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" event={"ID":"76c15550-c70e-4fce-a797-3cef528070db","Type":"ContainerStarted","Data":"55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a"}
Sep 10 22:31:42.895648 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.895475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" event={"ID":"76c15550-c70e-4fce-a797-3cef528070db","Type":"ContainerStarted","Data":"3e7ab5c6b013c817dade40b3f85e9b8edd80d6f7c337c74f44dc4df2f28e012d"}
Sep 10 22:31:42.916248 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:42.916190    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podStartSLOduration=0.916170451 podStartE2EDuration="916.170451ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:31:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:31:42.914841287 +0000 UTC m=+1572.299346712" watchObservedRunningTime="2026-09-10 22:31:42.916170451 +0000 UTC m=+1572.300675876"
Sep 10 22:31:43.431205 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:43.431173    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:43.435747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:43.435724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:43.900375 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:43.900342    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:43.901346 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:43.901322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:31:46.550791 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.550751    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"]
Sep 10 22:31:46.551365 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.551345    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7634911c-2073-4b6d-a8e0-af7fe7167434" containerName="istio-proxy"
Sep 10 22:31:46.554510 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.554490    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:46.556785 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.556762    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-9b2092\""
Sep 10 22:31:46.556880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.556771    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-9b2092-dockercfg-fxhb6\""
Sep 10 22:31:46.564085 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.564061    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"]
Sep 10 22:31:46.652712 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.652677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600fbe8c-42f5-4766-801d-d51c965e71b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:46.652898 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.652813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn6tw\" (UniqueName: \"kubernetes.io/projected/600fbe8c-42f5-4766-801d-d51c965e71b4-kube-api-access-wn6tw\") pod \"payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:46.754424 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.754365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600fbe8c-42f5-4766-801d-d51c965e71b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:46.754637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.754504    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn6tw\" (UniqueName: \"kubernetes.io/projected/600fbe8c-42f5-4766-801d-d51c965e71b4-kube-api-access-wn6tw\") pod \"payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:46.755076 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.755056    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600fbe8c-42f5-4766-801d-d51c965e71b4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:46.761744 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.761717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn6tw\" (UniqueName: \"kubernetes.io/projected/600fbe8c-42f5-4766-801d-d51c965e71b4-kube-api-access-wn6tw\") pod \"payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:46.865498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:46.865404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:47.138706 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:47.138678    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"]
Sep 10 22:31:47.140407 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:31:47.140373    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod600fbe8c_42f5_4766_801d_d51c965e71b4.slice/crio-121ad46310b9e11507a8878556bac7fc537351390635267ac8a6018cb753ba53 WatchSource:0}: Error finding container 121ad46310b9e11507a8878556bac7fc537351390635267ac8a6018cb753ba53: Status 404 returned error can't find the container with id 121ad46310b9e11507a8878556bac7fc537351390635267ac8a6018cb753ba53
Sep 10 22:31:47.922925 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:47.922891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4" event={"ID":"600fbe8c-42f5-4766-801d-d51c965e71b4","Type":"ContainerStarted","Data":"350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462"}
Sep 10 22:31:47.923372 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:47.922941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4" event={"ID":"600fbe8c-42f5-4766-801d-d51c965e71b4","Type":"ContainerStarted","Data":"121ad46310b9e11507a8878556bac7fc537351390635267ac8a6018cb753ba53"}
Sep 10 22:31:47.923372 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:47.922981    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:47.938830 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:47.938783    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4" podStartSLOduration=1.9387682000000002 podStartE2EDuration="1.9387682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:31:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:31:47.937102363 +0000 UTC m=+1577.321607788" watchObservedRunningTime="2026-09-10 22:31:47.9387682 +0000 UTC m=+1577.323273659"
Sep 10 22:31:50.510065 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:50.510028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:31:58.929510 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:58.929480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:31:59.941596 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:59.941563    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 22:31:59.941972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:31:59.941633    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 22:32:00.991731 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:00.991702    2563 generic.go:356] "Generic (PLEG): container finished" podID="76c15550-c70e-4fce-a797-3cef528070db" containerID="55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a" exitCode=0
Sep 10 22:32:00.992269 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:00.991789    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" event={"ID":"76c15550-c70e-4fce-a797-3cef528070db","Type":"ContainerDied","Data":"55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a"}
Sep 10 22:32:00.992343 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:00.992297    2563 scope.go:117] "RemoveContainer" containerID="55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a"
Sep 10 22:32:00.993604 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:00.993568    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:32:00.993739 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:00.993652    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:32:00.993739 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:00.993703    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"}
Sep 10 22:32:01.999231 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:01.999200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" event={"ID":"76c15550-c70e-4fce-a797-3cef528070db","Type":"ContainerStarted","Data":"35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c"}
Sep 10 22:32:02.431527 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:02.431405    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:32:02.431748 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:02.431718    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 10 22:32:02.431833 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:02.431783    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 10 22:32:03.431563 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:03.431517    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 10 22:32:03.432065 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:03.431587    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 10 22:32:04.431834 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:04.431797    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 10 22:32:04.432426 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:04.432392    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 10 22:32:05.432048 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:05.431998    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 10 22:32:05.432605 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:05.432070    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 10 22:32:06.431505 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:06.431463    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Sep 10 22:32:06.431720 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:06.431546    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Sep 10 22:32:07.435456 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:07.435417    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:32:07.435836 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:07.435677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:32:07.436545 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:07.436524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:32:14.976616 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:14.976578    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4_600fbe8c-42f5-4766-801d-d51c965e71b4/payload-processing/0.log"
Sep 10 22:32:20.210470 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.210425    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"]
Sep 10 22:32:20.212910 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.212891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.215893 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.215863    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-25241d\"/\"openshift-service-ca.crt\""
Sep 10 22:32:20.215893 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.215865    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-25241d\"/\"kube-root-ca.crt\""
Sep 10 22:32:20.216095 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.215865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-25241d\"/\"default-dockercfg-5pmxd\""
Sep 10 22:32:20.216095 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.215936    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-25241d\"/\"ipp-route-25241d-kserve-self-signed-certs\""
Sep 10 22:32:20.225430 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.225401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"]
Sep 10 22:32:20.270037 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.270003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31e4dbfc-5040-465c-a706-7c7905f6039c-tls-certs\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.270216 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.270059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-model-cache\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.270216 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.270090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-dshm\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.270216 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.270162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-home\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.270401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.270219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt95d\" (UniqueName: \"kubernetes.io/projected/31e4dbfc-5040-465c-a706-7c7905f6039c-kube-api-access-qt95d\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.270401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.270250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-tmp-dir\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.370983 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.370945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt95d\" (UniqueName: \"kubernetes.io/projected/31e4dbfc-5040-465c-a706-7c7905f6039c-kube-api-access-qt95d\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.370998    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-tmp-dir\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.371099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31e4dbfc-5040-465c-a706-7c7905f6039c-tls-certs\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.371136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-model-cache\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.371156    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-dshm\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371402 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.371200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-home\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.371481    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-tmp-dir\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.371546    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-model-cache\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.371608 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.371596    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-home\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.373510 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.373493    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-dshm\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.373640 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.373623    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31e4dbfc-5040-465c-a706-7c7905f6039c-tls-certs\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.378637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.378616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt95d\" (UniqueName: \"kubernetes.io/projected/31e4dbfc-5040-465c-a706-7c7905f6039c-kube-api-access-qt95d\") pod \"ipp-route-25241d-kserve-687f96d9f5-snqcj\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") " pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.522234 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.522196    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:20.795888 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:20.795865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"]
Sep 10 22:32:20.797638 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:32:20.797615    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31e4dbfc_5040_465c_a706_7c7905f6039c.slice/crio-a304a3e57c4636cd7919bd9706c478c1b5d6788f980db4296d889f976d2da98d WatchSource:0}: Error finding container a304a3e57c4636cd7919bd9706c478c1b5d6788f980db4296d889f976d2da98d: Status 404 returned error can't find the container with id a304a3e57c4636cd7919bd9706c478c1b5d6788f980db4296d889f976d2da98d
Sep 10 22:32:21.102199 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:21.102090    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj" event={"ID":"31e4dbfc-5040-465c-a706-7c7905f6039c","Type":"ContainerStarted","Data":"1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db"}
Sep 10 22:32:21.102199 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:21.102147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj" event={"ID":"31e4dbfc-5040-465c-a706-7c7905f6039c","Type":"ContainerStarted","Data":"a304a3e57c4636cd7919bd9706c478c1b5d6788f980db4296d889f976d2da98d"}
Sep 10 22:32:21.121515 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:21.121464    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj" podStartSLOduration=1.121433345 podStartE2EDuration="1.121433345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:32:21.120510855 +0000 UTC m=+1610.505016281" watchObservedRunningTime="2026-09-10 22:32:21.121433345 +0000 UTC m=+1610.505938769"
Sep 10 22:32:24.103405 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:24.103311    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:32:30.522614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:30.522575    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:30.523169 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:30.522628    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:30.535282 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:30.535256    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:31.164898 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:31.164870    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:32:39.000962 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:39.000927    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:32:45.620092 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:45.620058    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4_600fbe8c-42f5-4766-801d-d51c965e71b4/payload-processing/0.log"
Sep 10 22:32:50.621779 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.621731    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"]
Sep 10 22:32:50.624327 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.624300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:50.628415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.628389    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-5567cc\""
Sep 10 22:32:50.628581 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.628458    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-5567cc-dockercfg-7f542\""
Sep 10 22:32:50.636096 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.636071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89afae13-3808-408e-b64a-782597671e19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:50.636215 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.636170    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7dcs\" (UniqueName: \"kubernetes.io/projected/89afae13-3808-408e-b64a-782597671e19-kube-api-access-t7dcs\") pod \"payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:50.642131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.642092    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"]
Sep 10 22:32:50.737482 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.737170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7dcs\" (UniqueName: \"kubernetes.io/projected/89afae13-3808-408e-b64a-782597671e19-kube-api-access-t7dcs\") pod \"payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:50.737482 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.737229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89afae13-3808-408e-b64a-782597671e19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:50.740099 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.738196    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89afae13-3808-408e-b64a-782597671e19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:50.746371 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.746342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7dcs\" (UniqueName: \"kubernetes.io/projected/89afae13-3808-408e-b64a-782597671e19-kube-api-access-t7dcs\") pod \"payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:50.935107 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:50.935021    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:51.204019 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:51.203997    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"]
Sep 10 22:32:51.206240 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:32:51.206212    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89afae13_3808_408e_b64a_782597671e19.slice/crio-b17236188f81f26af2b2ddf9c13d7b40960d2f56ff43bd16e71317ec031a3b4c WatchSource:0}: Error finding container b17236188f81f26af2b2ddf9c13d7b40960d2f56ff43bd16e71317ec031a3b4c: Status 404 returned error can't find the container with id b17236188f81f26af2b2ddf9c13d7b40960d2f56ff43bd16e71317ec031a3b4c
Sep 10 22:32:51.253277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:51.253248    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn" event={"ID":"89afae13-3808-408e-b64a-782597671e19","Type":"ContainerStarted","Data":"b17236188f81f26af2b2ddf9c13d7b40960d2f56ff43bd16e71317ec031a3b4c"}
Sep 10 22:32:52.259889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:52.259850    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn" event={"ID":"89afae13-3808-408e-b64a-782597671e19","Type":"ContainerStarted","Data":"1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380"}
Sep 10 22:32:52.260360 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:52.260004    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:32:52.276320 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:52.276272    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn" podStartSLOduration=2.276252844 podStartE2EDuration="2.276252844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:32:52.274624845 +0000 UTC m=+1641.659130271" watchObservedRunningTime="2026-09-10 22:32:52.276252844 +0000 UTC m=+1641.660758269"
Sep 10 22:32:58.206791 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:32:58.206756    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:33:03.267889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:03.267854    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:33:14.369043 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.369007    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"]
Sep 10 22:33:14.371938 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.371920    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"
Sep 10 22:33:14.373852 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.373836    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 22:33:14.373945 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.373865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-mlfhj\""
Sep 10 22:33:14.379769 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.379750    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"]
Sep 10 22:33:14.458131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.458092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899ddae-25e1-4106-ad0c-e6d59abde486-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59\" (UID: \"4899ddae-25e1-4106-ad0c-e6d59abde486\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"
Sep 10 22:33:14.558738 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.558692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899ddae-25e1-4106-ad0c-e6d59abde486-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59\" (UID: \"4899ddae-25e1-4106-ad0c-e6d59abde486\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"
Sep 10 22:33:14.559291 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.559273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899ddae-25e1-4106-ad0c-e6d59abde486-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59\" (UID: \"4899ddae-25e1-4106-ad0c-e6d59abde486\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"
Sep 10 22:33:14.683356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.683269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"
Sep 10 22:33:14.953114 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:14.953091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"]
Sep 10 22:33:14.955877 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:33:14.955849    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4899ddae_25e1_4106_ad0c_e6d59abde486.slice/crio-24f555dd0bc6f73c1a840ac88a384f9b6399f306886eae330ad98eae7c66eadd WatchSource:0}: Error finding container 24f555dd0bc6f73c1a840ac88a384f9b6399f306886eae330ad98eae7c66eadd: Status 404 returned error can't find the container with id 24f555dd0bc6f73c1a840ac88a384f9b6399f306886eae330ad98eae7c66eadd
Sep 10 22:33:15.378729 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:15.378698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59" event={"ID":"4899ddae-25e1-4106-ad0c-e6d59abde486","Type":"ContainerStarted","Data":"bb0e947fb3a4dd78b56b2677bf843ed61f4b1db9cf9b2a654ff8be6bcce1b461"}
Sep 10 22:33:15.379115 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:15.378750    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59" event={"ID":"4899ddae-25e1-4106-ad0c-e6d59abde486","Type":"ContainerStarted","Data":"24f555dd0bc6f73c1a840ac88a384f9b6399f306886eae330ad98eae7c66eadd"}
Sep 10 22:33:15.394333 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:15.394287    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59" podStartSLOduration=1.3942722330000001 podStartE2EDuration="1.394272233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:33:15.392428791 +0000 UTC m=+1664.776934217" watchObservedRunningTime="2026-09-10 22:33:15.394272233 +0000 UTC m=+1664.778777683"
Sep 10 22:33:16.386867 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:16.386828    2563 generic.go:356] "Generic (PLEG): container finished" podID="4899ddae-25e1-4106-ad0c-e6d59abde486" containerID="bb0e947fb3a4dd78b56b2677bf843ed61f4b1db9cf9b2a654ff8be6bcce1b461" exitCode=0
Sep 10 22:33:16.387295 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:16.386919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59" event={"ID":"4899ddae-25e1-4106-ad0c-e6d59abde486","Type":"ContainerDied","Data":"bb0e947fb3a4dd78b56b2677bf843ed61f4b1db9cf9b2a654ff8be6bcce1b461"}
Sep 10 22:33:17.524166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:17.524142    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"
Sep 10 22:33:17.685340 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:17.685262    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899ddae-25e1-4106-ad0c-e6d59abde486-maas-api-service-ca\") pod \"4899ddae-25e1-4106-ad0c-e6d59abde486\" (UID: \"4899ddae-25e1-4106-ad0c-e6d59abde486\") "
Sep 10 22:33:17.685675 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:17.685654    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4899ddae-25e1-4106-ad0c-e6d59abde486-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4899ddae-25e1-4106-ad0c-e6d59abde486" (UID: "4899ddae-25e1-4106-ad0c-e6d59abde486"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:33:17.786533 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:17.786505    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4899ddae-25e1-4106-ad0c-e6d59abde486-maas-api-service-ca\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:33:18.398475 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:18.398447    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"
Sep 10 22:33:18.398639 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:18.398501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59" event={"ID":"4899ddae-25e1-4106-ad0c-e6d59abde486","Type":"ContainerDied","Data":"24f555dd0bc6f73c1a840ac88a384f9b6399f306886eae330ad98eae7c66eadd"}
Sep 10 22:33:18.398639 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:18.398547    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24f555dd0bc6f73c1a840ac88a384f9b6399f306886eae330ad98eae7c66eadd"
Sep 10 22:33:20.609472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:20.608461    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"]
Sep 10 22:33:20.609472 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:20.608811    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn" podUID="89afae13-3808-408e-b64a-782597671e19" containerName="payload-pre-processing" containerID="cri-o://1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380" gracePeriod=30
Sep 10 22:33:20.755989 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:20.755934    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"]
Sep 10 22:33:20.758048 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:20.758017    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-5567cc-7fc59"]
Sep 10 22:33:20.867670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:20.867596    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:33:21.015771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.015734    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7dcs\" (UniqueName: \"kubernetes.io/projected/89afae13-3808-408e-b64a-782597671e19-kube-api-access-t7dcs\") pod \"89afae13-3808-408e-b64a-782597671e19\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") "
Sep 10 22:33:21.015968 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.015779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89afae13-3808-408e-b64a-782597671e19-plugins-config-volume\") pod \"89afae13-3808-408e-b64a-782597671e19\" (UID: \"89afae13-3808-408e-b64a-782597671e19\") "
Sep 10 22:33:21.016160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.016136    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89afae13-3808-408e-b64a-782597671e19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89afae13-3808-408e-b64a-782597671e19" (UID: "89afae13-3808-408e-b64a-782597671e19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:33:21.018014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.017991    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89afae13-3808-408e-b64a-782597671e19-kube-api-access-t7dcs" (OuterVolumeSpecName: "kube-api-access-t7dcs") pod "89afae13-3808-408e-b64a-782597671e19" (UID: "89afae13-3808-408e-b64a-782597671e19"). InnerVolumeSpecName "kube-api-access-t7dcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:33:21.111397 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.111363    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4899ddae-25e1-4106-ad0c-e6d59abde486" path="/var/lib/kubelet/pods/4899ddae-25e1-4106-ad0c-e6d59abde486/volumes"
Sep 10 22:33:21.116854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.116833    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7dcs\" (UniqueName: \"kubernetes.io/projected/89afae13-3808-408e-b64a-782597671e19-kube-api-access-t7dcs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:33:21.116927 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.116858    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89afae13-3808-408e-b64a-782597671e19-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:33:21.414659 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.414578    2563 generic.go:356] "Generic (PLEG): container finished" podID="89afae13-3808-408e-b64a-782597671e19" containerID="1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380" exitCode=0
Sep 10 22:33:21.414659 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.414623    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"
Sep 10 22:33:21.414831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.414672    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn" event={"ID":"89afae13-3808-408e-b64a-782597671e19","Type":"ContainerDied","Data":"1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380"}
Sep 10 22:33:21.414831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.414719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn" event={"ID":"89afae13-3808-408e-b64a-782597671e19","Type":"ContainerDied","Data":"b17236188f81f26af2b2ddf9c13d7b40960d2f56ff43bd16e71317ec031a3b4c"}
Sep 10 22:33:21.414831 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.414748    2563 scope.go:117] "RemoveContainer" containerID="1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380"
Sep 10 22:33:21.423526 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.423507    2563 scope.go:117] "RemoveContainer" containerID="1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380"
Sep 10 22:33:21.423799 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:33:21.423780    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380\": container with ID starting with 1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380 not found: ID does not exist" containerID="1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380"
Sep 10 22:33:21.423852 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.423808    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380"} err="failed to get container status \"1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380\": rpc error: code = NotFound desc = could not find container \"1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380\": container with ID starting with 1040264846c75bdfe5e4bdcb5633a4bb4f8938018104cd87fc43b296f2f0d380 not found: ID does not exist"
Sep 10 22:33:21.431624 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.431601    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"]
Sep 10 22:33:21.435413 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:21.435393    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-5567cc-77d5f48cc4-4r6rn"]
Sep 10 22:33:23.110558 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:23.110522    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89afae13-3808-408e-b64a-782597671e19" path="/var/lib/kubelet/pods/89afae13-3808-408e-b64a-782597671e19/volumes"
Sep 10 22:33:25.103297 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:25.103264    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:33:43.802157 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:43.802119    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:33:51.898967 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:51.898875    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"]
Sep 10 22:33:57.001770 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:33:57.001732    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:34:06.918108 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:06.918037    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj" podUID="31e4dbfc-5040-465c-a706-7c7905f6039c" containerName="main" containerID="cri-o://1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db" gracePeriod=45
Sep 10 22:34:07.170689 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.170634    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:34:07.343589 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.343557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-model-cache\") pod \"31e4dbfc-5040-465c-a706-7c7905f6039c\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") "
Sep 10 22:34:07.343748 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.343618    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31e4dbfc-5040-465c-a706-7c7905f6039c-tls-certs\") pod \"31e4dbfc-5040-465c-a706-7c7905f6039c\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") "
Sep 10 22:34:07.343748 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.343713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt95d\" (UniqueName: \"kubernetes.io/projected/31e4dbfc-5040-465c-a706-7c7905f6039c-kube-api-access-qt95d\") pod \"31e4dbfc-5040-465c-a706-7c7905f6039c\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") "
Sep 10 22:34:07.343858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.343748    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-tmp-dir\") pod \"31e4dbfc-5040-465c-a706-7c7905f6039c\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") "
Sep 10 22:34:07.343858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.343777    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-home\") pod \"31e4dbfc-5040-465c-a706-7c7905f6039c\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") "
Sep 10 22:34:07.343858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.343806    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-dshm\") pod \"31e4dbfc-5040-465c-a706-7c7905f6039c\" (UID: \"31e4dbfc-5040-465c-a706-7c7905f6039c\") "
Sep 10 22:34:07.343858 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.343810    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-model-cache" (OuterVolumeSpecName: "model-cache") pod "31e4dbfc-5040-465c-a706-7c7905f6039c" (UID: "31e4dbfc-5040-465c-a706-7c7905f6039c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:34:07.344081 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.344024    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "31e4dbfc-5040-465c-a706-7c7905f6039c" (UID: "31e4dbfc-5040-465c-a706-7c7905f6039c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:34:07.344081 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.344032    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-home" (OuterVolumeSpecName: "home") pod "31e4dbfc-5040-465c-a706-7c7905f6039c" (UID: "31e4dbfc-5040-465c-a706-7c7905f6039c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:34:07.344187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.344107    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:34:07.344187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.344126    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:34:07.344187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.344136    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:34:07.346069 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.346039    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-dshm" (OuterVolumeSpecName: "dshm") pod "31e4dbfc-5040-465c-a706-7c7905f6039c" (UID: "31e4dbfc-5040-465c-a706-7c7905f6039c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:34:07.346201 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.346178    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e4dbfc-5040-465c-a706-7c7905f6039c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "31e4dbfc-5040-465c-a706-7c7905f6039c" (UID: "31e4dbfc-5040-465c-a706-7c7905f6039c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:34:07.346243 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.346195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31e4dbfc-5040-465c-a706-7c7905f6039c-kube-api-access-qt95d" (OuterVolumeSpecName: "kube-api-access-qt95d") pod "31e4dbfc-5040-465c-a706-7c7905f6039c" (UID: "31e4dbfc-5040-465c-a706-7c7905f6039c"). InnerVolumeSpecName "kube-api-access-qt95d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:34:07.445316 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.445246    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt95d\" (UniqueName: \"kubernetes.io/projected/31e4dbfc-5040-465c-a706-7c7905f6039c-kube-api-access-qt95d\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:34:07.445316 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.445271    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31e4dbfc-5040-465c-a706-7c7905f6039c-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:34:07.445316 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.445281    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31e4dbfc-5040-465c-a706-7c7905f6039c-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:34:07.656854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.656815    2563 generic.go:356] "Generic (PLEG): container finished" podID="31e4dbfc-5040-465c-a706-7c7905f6039c" containerID="1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db" exitCode=0
Sep 10 22:34:07.657025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.656862    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"
Sep 10 22:34:07.657025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.656881    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj" event={"ID":"31e4dbfc-5040-465c-a706-7c7905f6039c","Type":"ContainerDied","Data":"1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db"}
Sep 10 22:34:07.657025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.656920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj" event={"ID":"31e4dbfc-5040-465c-a706-7c7905f6039c","Type":"ContainerDied","Data":"a304a3e57c4636cd7919bd9706c478c1b5d6788f980db4296d889f976d2da98d"}
Sep 10 22:34:07.657025 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.656943    2563 scope.go:117] "RemoveContainer" containerID="1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db"
Sep 10 22:34:07.665938 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.665922    2563 scope.go:117] "RemoveContainer" containerID="1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db"
Sep 10 22:34:07.666203 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:34:07.666181    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db\": container with ID starting with 1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db not found: ID does not exist" containerID="1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db"
Sep 10 22:34:07.666257 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.666212    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db"} err="failed to get container status \"1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db\": rpc error: code = NotFound desc = could not find container \"1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db\": container with ID starting with 1001466376ec19feebbf7318c5ba21de381f57ead34d09ceeb39737ce16466db not found: ID does not exist"
Sep 10 22:34:07.677084 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.677063    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"]
Sep 10 22:34:07.680984 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:07.680965    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-25241d/ipp-route-25241d-kserve-687f96d9f5-snqcj"]
Sep 10 22:34:09.110554 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:34:09.110520    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31e4dbfc-5040-465c-a706-7c7905f6039c" path="/var/lib/kubelet/pods/31e4dbfc-5040-465c-a706-7c7905f6039c/volumes"
Sep 10 22:35:28.168273 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.168182    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"]
Sep 10 22:35:28.168802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.168455    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4" podUID="600fbe8c-42f5-4766-801d-d51c965e71b4" containerName="payload-processing" containerID="cri-o://350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462" gracePeriod=30
Sep 10 22:35:28.433934 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.433902    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:35:28.566533 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.566497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600fbe8c-42f5-4766-801d-d51c965e71b4-plugins-config-volume\") pod \"600fbe8c-42f5-4766-801d-d51c965e71b4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") "
Sep 10 22:35:28.566704 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.566561    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn6tw\" (UniqueName: \"kubernetes.io/projected/600fbe8c-42f5-4766-801d-d51c965e71b4-kube-api-access-wn6tw\") pod \"600fbe8c-42f5-4766-801d-d51c965e71b4\" (UID: \"600fbe8c-42f5-4766-801d-d51c965e71b4\") "
Sep 10 22:35:28.566866 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.566839    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/600fbe8c-42f5-4766-801d-d51c965e71b4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "600fbe8c-42f5-4766-801d-d51c965e71b4" (UID: "600fbe8c-42f5-4766-801d-d51c965e71b4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:35:28.568731 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.568709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600fbe8c-42f5-4766-801d-d51c965e71b4-kube-api-access-wn6tw" (OuterVolumeSpecName: "kube-api-access-wn6tw") pod "600fbe8c-42f5-4766-801d-d51c965e71b4" (UID: "600fbe8c-42f5-4766-801d-d51c965e71b4"). InnerVolumeSpecName "kube-api-access-wn6tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:35:28.667795 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.667764    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/600fbe8c-42f5-4766-801d-d51c965e71b4-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:28.667795 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:28.667791    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn6tw\" (UniqueName: \"kubernetes.io/projected/600fbe8c-42f5-4766-801d-d51c965e71b4-kube-api-access-wn6tw\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:29.059996 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.059960    2563 generic.go:356] "Generic (PLEG): container finished" podID="600fbe8c-42f5-4766-801d-d51c965e71b4" containerID="350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462" exitCode=0
Sep 10 22:35:29.060167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.060006    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"
Sep 10 22:35:29.060167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.060043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4" event={"ID":"600fbe8c-42f5-4766-801d-d51c965e71b4","Type":"ContainerDied","Data":"350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462"}
Sep 10 22:35:29.060167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.060096    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4" event={"ID":"600fbe8c-42f5-4766-801d-d51c965e71b4","Type":"ContainerDied","Data":"121ad46310b9e11507a8878556bac7fc537351390635267ac8a6018cb753ba53"}
Sep 10 22:35:29.060167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.060120    2563 scope.go:117] "RemoveContainer" containerID="350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462"
Sep 10 22:35:29.075143 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.075123    2563 scope.go:117] "RemoveContainer" containerID="350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462"
Sep 10 22:35:29.075449 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:35:29.075414    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462\": container with ID starting with 350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462 not found: ID does not exist" containerID="350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462"
Sep 10 22:35:29.075508 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.075459    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462"} err="failed to get container status \"350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462\": rpc error: code = NotFound desc = could not find container \"350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462\": container with ID starting with 350bd25434147849ea4e2447e321765c454ce9a16d8be0b06216b65c69183462 not found: ID does not exist"
Sep 10 22:35:29.088663 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.088632    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"]
Sep 10 22:35:29.090740 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.090718    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-9b2092-5ffffddc87-rqvd4"]
Sep 10 22:35:29.110935 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:29.110909    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600fbe8c-42f5-4766-801d-d51c965e71b4" path="/var/lib/kubelet/pods/600fbe8c-42f5-4766-801d-d51c965e71b4/volumes"
Sep 10 22:35:31.611722 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:31.611685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:35:34.665536 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:34.665498    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"]
Sep 10 22:35:34.665974 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:34.665810    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" containerID="cri-o://35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c" gracePeriod=30
Sep 10 22:35:37.438251 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:37.438215    2563 patch_prober.go:28] interesting pod/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 22:35:37.438733 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:37.438282    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 22:35:39.915715 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:39.915691    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:35:40.071331 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071293    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96nlq\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-kube-api-access-96nlq\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071561 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071344    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c15550-c70e-4fce-a797-3cef528070db-istiod-ca-cert\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071561 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071379    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-istio-token\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071561 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-envoy\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071561 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071476    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-socket\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-certs\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071649    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-credential-socket\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-data\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.071790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071777    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c15550-c70e-4fce-a797-3cef528070db-istio-podinfo\") pod \"76c15550-c70e-4fce-a797-3cef528070db\" (UID: \"76c15550-c70e-4fce-a797-3cef528070db\") "
Sep 10 22:35:40.072004 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071790    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:40.072004 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:40.072004 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.071888    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c15550-c70e-4fce-a797-3cef528070db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:35:40.072166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.072006    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:40.072166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.072084    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-data" (OuterVolumeSpecName: "istio-data") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:40.072166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.072151    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c15550-c70e-4fce-a797-3cef528070db-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.072300 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.072167    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.072300 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.072180    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.072300 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.072195    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.073794 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.073771    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:40.073882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.073840    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-kube-api-access-96nlq" (OuterVolumeSpecName: "kube-api-access-96nlq") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "kube-api-access-96nlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:35:40.074361 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.074344    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-istio-token" (OuterVolumeSpecName: "istio-token") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:35:40.074411 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.074360    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76c15550-c70e-4fce-a797-3cef528070db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76c15550-c70e-4fce-a797-3cef528070db" (UID: "76c15550-c70e-4fce-a797-3cef528070db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:35:40.116504 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.116466    2563 generic.go:356] "Generic (PLEG): container finished" podID="76c15550-c70e-4fce-a797-3cef528070db" containerID="35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c" exitCode=0
Sep 10 22:35:40.116645 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.116507    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"
Sep 10 22:35:40.116645 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.116513    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" event={"ID":"76c15550-c70e-4fce-a797-3cef528070db","Type":"ContainerDied","Data":"35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c"}
Sep 10 22:35:40.116645 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.116554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn" event={"ID":"76c15550-c70e-4fce-a797-3cef528070db","Type":"ContainerDied","Data":"3e7ab5c6b013c817dade40b3f85e9b8edd80d6f7c337c74f44dc4df2f28e012d"}
Sep 10 22:35:40.116645 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.116576    2563 scope.go:117] "RemoveContainer" containerID="35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c"
Sep 10 22:35:40.126263 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.126238    2563 scope.go:117] "RemoveContainer" containerID="55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a"
Sep 10 22:35:40.134378 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.134357    2563 scope.go:117] "RemoveContainer" containerID="35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c"
Sep 10 22:35:40.134666 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:35:40.134647    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c\": container with ID starting with 35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c not found: ID does not exist" containerID="35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c"
Sep 10 22:35:40.134737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.134672    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c"} err="failed to get container status \"35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c\": rpc error: code = NotFound desc = could not find container \"35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c\": container with ID starting with 35943241335fb4543319404285a692b1638c79c006649df09b73848e03b1860c not found: ID does not exist"
Sep 10 22:35:40.134737 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.134690    2563 scope.go:117] "RemoveContainer" containerID="55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a"
Sep 10 22:35:40.134929 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:35:40.134911    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a\": container with ID starting with 55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a not found: ID does not exist" containerID="55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a"
Sep 10 22:35:40.134966 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.134935    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a"} err="failed to get container status \"55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a\": rpc error: code = NotFound desc = could not find container \"55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a\": container with ID starting with 55a59da01b95543f0b6973dc21a4207947adb43fb840e081ecd6e52a0052e47a not found: ID does not exist"
Sep 10 22:35:40.138650 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.138619    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"]
Sep 10 22:35:40.142334 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.142314    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-9b2092-openshift-default-78b6b5784-mtjrn"]
Sep 10 22:35:40.172870 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.172845    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96nlq\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-kube-api-access-96nlq\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.172870 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.172869    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c15550-c70e-4fce-a797-3cef528070db-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.172994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.172878    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.172994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.172886    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c15550-c70e-4fce-a797-3cef528070db-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:40.172994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:40.172893    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c15550-c70e-4fce-a797-3cef528070db-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:41.110574 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:41.110542    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76c15550-c70e-4fce-a797-3cef528070db" path="/var/lib/kubelet/pods/76c15550-c70e-4fce-a797-3cef528070db/volumes"
Sep 10 22:35:45.456542 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.456502    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"]
Sep 10 22:35:45.458477 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.458453    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-e367c1-29817990-nmvxr"]
Sep 10 22:35:45.482683 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.482650    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"]
Sep 10 22:35:45.482951 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.482921    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs" podUID="69e42a52-81e6-4a82-ae0a-6af1da7794b6" containerName="payload-processing" containerID="cri-o://bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d" gracePeriod=30
Sep 10 22:35:45.741929 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.741907    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:35:45.820091 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.819675    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbh4h\" (UniqueName: \"kubernetes.io/projected/69e42a52-81e6-4a82-ae0a-6af1da7794b6-kube-api-access-rbh4h\") pod \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") "
Sep 10 22:35:45.820091 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.819781    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e42a52-81e6-4a82-ae0a-6af1da7794b6-plugins-config-volume\") pod \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\" (UID: \"69e42a52-81e6-4a82-ae0a-6af1da7794b6\") "
Sep 10 22:35:45.820844 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.820715    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69e42a52-81e6-4a82-ae0a-6af1da7794b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69e42a52-81e6-4a82-ae0a-6af1da7794b6" (UID: "69e42a52-81e6-4a82-ae0a-6af1da7794b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:35:45.823334 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.823299    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69e42a52-81e6-4a82-ae0a-6af1da7794b6-kube-api-access-rbh4h" (OuterVolumeSpecName: "kube-api-access-rbh4h") pod "69e42a52-81e6-4a82-ae0a-6af1da7794b6" (UID: "69e42a52-81e6-4a82-ae0a-6af1da7794b6"). InnerVolumeSpecName "kube-api-access-rbh4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:35:45.921390 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.921359    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbh4h\" (UniqueName: \"kubernetes.io/projected/69e42a52-81e6-4a82-ae0a-6af1da7794b6-kube-api-access-rbh4h\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:45.921390 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:45.921386    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69e42a52-81e6-4a82-ae0a-6af1da7794b6-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:46.150018 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.149986    2563 generic.go:356] "Generic (PLEG): container finished" podID="69e42a52-81e6-4a82-ae0a-6af1da7794b6" containerID="bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d" exitCode=0
Sep 10 22:35:46.150242 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.150038    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"
Sep 10 22:35:46.150242 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.150083    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs" event={"ID":"69e42a52-81e6-4a82-ae0a-6af1da7794b6","Type":"ContainerDied","Data":"bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d"}
Sep 10 22:35:46.150242 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.150143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs" event={"ID":"69e42a52-81e6-4a82-ae0a-6af1da7794b6","Type":"ContainerDied","Data":"b858280dedcdb162266231a4f7415aa974e964d7b13ed7dfc25bc26b5d219820"}
Sep 10 22:35:46.150242 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.150165    2563 scope.go:117] "RemoveContainer" containerID="bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d"
Sep 10 22:35:46.158797 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.158781    2563 scope.go:117] "RemoveContainer" containerID="bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d"
Sep 10 22:35:46.159120 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:35:46.159101    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d\": container with ID starting with bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d not found: ID does not exist" containerID="bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d"
Sep 10 22:35:46.159170 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.159128    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d"} err="failed to get container status \"bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d\": rpc error: code = NotFound desc = could not find container \"bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d\": container with ID starting with bca0f88b6e90141281ab7b86a1e7d118568e1387aaa374df6c2080684d1e304d not found: ID does not exist"
Sep 10 22:35:46.170038 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.170000    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"]
Sep 10 22:35:46.177233 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:46.177211    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-e367c1-cb679bdb9-c2sbs"]
Sep 10 22:35:47.110996 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:47.110962    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f37169d-7606-4139-8a27-8b6b4b671ac0" path="/var/lib/kubelet/pods/5f37169d-7606-4139-8a27-8b6b4b671ac0/volumes"
Sep 10 22:35:47.111356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:47.111303    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69e42a52-81e6-4a82-ae0a-6af1da7794b6" path="/var/lib/kubelet/pods/69e42a52-81e6-4a82-ae0a-6af1da7794b6/volumes"
Sep 10 22:35:48.206986 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:48.206952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:35:52.511303 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:52.511272    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"]
Sep 10 22:35:52.511848 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:52.511484    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" containerID="cri-o://eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381" gracePeriod=30
Sep 10 22:35:54.640357 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:54.640316    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 22:35:54.640771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:54.640388    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 22:35:57.761688 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.761663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:35:57.825401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-token\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825401 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825364    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-certs\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825405    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-credential-socket\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825429    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-socket\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825489    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-envoy\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-data\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825581    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istiod-ca-cert\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825613    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-podinfo\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.825664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825650    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fx7k\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-kube-api-access-9fx7k\") pod \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\" (UID: \"c022a573-7bd3-4afb-bb5f-7da6e320fff7\") "
Sep 10 22:35:57.826149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825768    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:57.826149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.825815    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:57.826149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.826035    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-credential-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.826149 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.826056    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-socket\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.826486 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.826284    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:57.826486 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.826329    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:35:57.826691 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.826631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-data" (OuterVolumeSpecName: "istio-data") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:57.827870 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.827825    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-token" (OuterVolumeSpecName: "istio-token") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:35:57.828410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.828376    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:35:57.828410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.828394    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-kube-api-access-9fx7k" (OuterVolumeSpecName: "kube-api-access-9fx7k") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "kube-api-access-9fx7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:35:57.828565 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.828389    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c022a573-7bd3-4afb-bb5f-7da6e320fff7" (UID: "c022a573-7bd3-4afb-bb5f-7da6e320fff7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 22:35:57.927064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.927032    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-data\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.927064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.927058    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istiod-ca-cert\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.927064 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.927066    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-podinfo\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.927286 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.927075    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fx7k\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-kube-api-access-9fx7k\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.927286 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.927084    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-token\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.927286 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.927092    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-workload-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:57.927286 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:57.927099    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c022a573-7bd3-4afb-bb5f-7da6e320fff7-istio-envoy\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:35:58.211345 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.211253    2563 generic.go:356] "Generic (PLEG): container finished" podID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerID="eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381" exitCode=0
Sep 10 22:35:58.211345 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.211305    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"
Sep 10 22:35:58.211345 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.211332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" event={"ID":"c022a573-7bd3-4afb-bb5f-7da6e320fff7","Type":"ContainerDied","Data":"eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381"}
Sep 10 22:35:58.211667 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.211389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg" event={"ID":"c022a573-7bd3-4afb-bb5f-7da6e320fff7","Type":"ContainerDied","Data":"deaa01a425b47018e7cfda99134fcbece3c04f15b53a8998785ba3fa901b1434"}
Sep 10 22:35:58.211667 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.211411    2563 scope.go:117] "RemoveContainer" containerID="eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381"
Sep 10 22:35:58.220473 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.220449    2563 scope.go:117] "RemoveContainer" containerID="ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab"
Sep 10 22:35:58.228083 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.228068    2563 scope.go:117] "RemoveContainer" containerID="eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381"
Sep 10 22:35:58.228321 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:35:58.228305    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381\": container with ID starting with eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381 not found: ID does not exist" containerID="eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381"
Sep 10 22:35:58.228368 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.228329    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381"} err="failed to get container status \"eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381\": rpc error: code = NotFound desc = could not find container \"eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381\": container with ID starting with eb8e9576c384c45f2ca20438c30cb4031ab97f422a5ed36ee60ce120f072e381 not found: ID does not exist"
Sep 10 22:35:58.228368 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.228344    2563 scope.go:117] "RemoveContainer" containerID="ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab"
Sep 10 22:35:58.228595 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:35:58.228574    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab\": container with ID starting with ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab not found: ID does not exist" containerID="ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab"
Sep 10 22:35:58.228638 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.228595    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab"} err="failed to get container status \"ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab\": rpc error: code = NotFound desc = could not find container \"ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab\": container with ID starting with ea73368b3016135e22c039ec633d815a73ab8f2be5017c7cf3ce200f475f58ab not found: ID does not exist"
Sep 10 22:35:58.232427 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.232408    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"]
Sep 10 22:35:58.240049 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:58.240025    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-e367c1-openshift-default-5d7488766-5wmvg"]
Sep 10 22:35:59.111311 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:35:59.111271    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" path="/var/lib/kubelet/pods/c022a573-7bd3-4afb-bb5f-7da6e320fff7/volumes"
Sep 10 22:36:03.356940 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.356904    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"]
Sep 10 22:36:03.359381 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.359355    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-0ada08-29817990-c2lx8"]
Sep 10 22:36:03.389072 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.389035    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"]
Sep 10 22:36:03.389341 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.389308    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp" podUID="e508224a-fcce-472b-a2cd-af03f0b71d04" containerName="payload-processing" containerID="cri-o://d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606" gracePeriod=30
Sep 10 22:36:03.394498 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.394467    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"]
Sep 10 22:36:03.394738 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.394705    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb" podUID="a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" containerName="payload-pre-processing" containerID="cri-o://8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c" gracePeriod=30
Sep 10 22:36:03.664832 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.664808    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:36:03.671840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.671815    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:36:03.778964 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.778928    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jrnf\" (UniqueName: \"kubernetes.io/projected/e508224a-fcce-472b-a2cd-af03f0b71d04-kube-api-access-4jrnf\") pod \"e508224a-fcce-472b-a2cd-af03f0b71d04\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") "
Sep 10 22:36:03.779167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.778983    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-plugins-config-volume\") pod \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") "
Sep 10 22:36:03.779167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.779021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508224a-fcce-472b-a2cd-af03f0b71d04-plugins-config-volume\") pod \"e508224a-fcce-472b-a2cd-af03f0b71d04\" (UID: \"e508224a-fcce-472b-a2cd-af03f0b71d04\") "
Sep 10 22:36:03.779167 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.779055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfnhl\" (UniqueName: \"kubernetes.io/projected/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-kube-api-access-dfnhl\") pod \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\" (UID: \"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c\") "
Sep 10 22:36:03.779351 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.779324    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" (UID: "a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:36:03.779422 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.779373    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e508224a-fcce-472b-a2cd-af03f0b71d04-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e508224a-fcce-472b-a2cd-af03f0b71d04" (UID: "e508224a-fcce-472b-a2cd-af03f0b71d04"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:36:03.781293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.781271    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e508224a-fcce-472b-a2cd-af03f0b71d04-kube-api-access-4jrnf" (OuterVolumeSpecName: "kube-api-access-4jrnf") pod "e508224a-fcce-472b-a2cd-af03f0b71d04" (UID: "e508224a-fcce-472b-a2cd-af03f0b71d04"). InnerVolumeSpecName "kube-api-access-4jrnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:36:03.781293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.781284    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-kube-api-access-dfnhl" (OuterVolumeSpecName: "kube-api-access-dfnhl") pod "a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" (UID: "a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c"). InnerVolumeSpecName "kube-api-access-dfnhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:36:03.880599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.880571    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e508224a-fcce-472b-a2cd-af03f0b71d04-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:36:03.880599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.880593    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfnhl\" (UniqueName: \"kubernetes.io/projected/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-kube-api-access-dfnhl\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:36:03.880599 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.880604    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jrnf\" (UniqueName: \"kubernetes.io/projected/e508224a-fcce-472b-a2cd-af03f0b71d04-kube-api-access-4jrnf\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:36:03.880787 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:03.880614    2563 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c-plugins-config-volume\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:36:04.240329 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.240292    2563 generic.go:356] "Generic (PLEG): container finished" podID="a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" containerID="8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c" exitCode=0
Sep 10 22:36:04.240544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.240335    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"
Sep 10 22:36:04.240544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.240380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb" event={"ID":"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c","Type":"ContainerDied","Data":"8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c"}
Sep 10 22:36:04.240544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.240428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb" event={"ID":"a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c","Type":"ContainerDied","Data":"b61edda208c70015dad0806497a4c9bfaec17bd59d3c8b69dccf3308eda93db3"}
Sep 10 22:36:04.240544 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.240487    2563 scope.go:117] "RemoveContainer" containerID="8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c"
Sep 10 22:36:04.241851 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.241819    2563 generic.go:356] "Generic (PLEG): container finished" podID="e508224a-fcce-472b-a2cd-af03f0b71d04" containerID="d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606" exitCode=0
Sep 10 22:36:04.241933 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.241873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp" event={"ID":"e508224a-fcce-472b-a2cd-af03f0b71d04","Type":"ContainerDied","Data":"d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606"}
Sep 10 22:36:04.241933 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.241905    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp" event={"ID":"e508224a-fcce-472b-a2cd-af03f0b71d04","Type":"ContainerDied","Data":"fb9832cbe1f969dc867635b045ae358376eb2c5f7113d978495271153bc9d7fd"}
Sep 10 22:36:04.241933 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.241917    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"
Sep 10 22:36:04.252880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.250760    2563 scope.go:117] "RemoveContainer" containerID="8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c"
Sep 10 22:36:04.252880 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:36:04.252732    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c\": container with ID starting with 8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c not found: ID does not exist" containerID="8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c"
Sep 10 22:36:04.252880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.252769    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c"} err="failed to get container status \"8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c\": rpc error: code = NotFound desc = could not find container \"8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c\": container with ID starting with 8e17d496631d6a2645d861c54e877f70b13b760bcc19c9a2cc282b4e417ff83c not found: ID does not exist"
Sep 10 22:36:04.252880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.252792    2563 scope.go:117] "RemoveContainer" containerID="d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606"
Sep 10 22:36:04.261691 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.261670    2563 scope.go:117] "RemoveContainer" containerID="d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606"
Sep 10 22:36:04.262009 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:36:04.261988    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606\": container with ID starting with d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606 not found: ID does not exist" containerID="d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606"
Sep 10 22:36:04.262072 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.262015    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606"} err="failed to get container status \"d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606\": rpc error: code = NotFound desc = could not find container \"d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606\": container with ID starting with d56c2a12d61c20ca0c2e133272c684752fa01bc7fe8f7f9dcd1103be601c6606 not found: ID does not exist"
Sep 10 22:36:04.276841 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.276817    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"]
Sep 10 22:36:04.281716 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.281694    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-0ada08-59c8d5bd5-d9lwp"]
Sep 10 22:36:04.291646 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.291626    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"]
Sep 10 22:36:04.295080 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:04.295060    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-0ada08-66999f6c4b-h8rcb"]
Sep 10 22:36:05.109726 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:05.109695    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" path="/var/lib/kubelet/pods/a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c/volumes"
Sep 10 22:36:05.110108 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:05.110036    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e508224a-fcce-472b-a2cd-af03f0b71d04" path="/var/lib/kubelet/pods/e508224a-fcce-472b-a2cd-af03f0b71d04/volumes"
Sep 10 22:36:05.110306 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:05.110292    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f21aaea6-7184-46eb-a2d3-00050465c831" path="/var/lib/kubelet/pods/f21aaea6-7184-46eb-a2d3-00050465c831/volumes"
Sep 10 22:36:06.109777 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:06.109745    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:36:15.000464 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:15.000417    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:36:31.441822 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:31.441789    2563 scope.go:117] "RemoveContainer" containerID="aa7979f86811b8e30531914f2a48d2b063173b92424892a6eaa802da8c1913c3"
Sep 10 22:36:31.449928 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:31.449903    2563 scope.go:117] "RemoveContainer" containerID="d8ec5d74a93e34a505c33524c4ada10b3c0e6484faa9ce5c21fe697b7aef769e"
Sep 10 22:36:31.457145 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:36:31.457128    2563 scope.go:117] "RemoveContainer" containerID="fc073ccb067eedfa0e40c79cb50fe68e82c8351605540b8889408808511bf9de"
Sep 10 22:37:37.152011 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:37:37.151976    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:38:00.756354 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:38:00.756320    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:38:08.518854 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:38:08.518813    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:38:15.521263 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:38:15.521227    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:38:23.149184 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:38:23.149098    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:38:37.604525 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:38:37.604483    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:38:44.008214 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:38:44.008182    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:39:31.511160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:39:31.511127    2563 scope.go:117] "RemoveContainer" containerID="bb0e947fb3a4dd78b56b2677bf843ed61f4b1db9cf9b2a654ff8be6bcce1b461"
Sep 10 22:40:09.103939 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:09.103902    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:40:16.902676 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:16.902642    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:40:22.700225 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:22.700189    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:40:23.494270 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494233    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"]
Sep 10 22:40:23.494771 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494751    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89afae13-3808-408e-b64a-782597671e19" containerName="payload-pre-processing"
Sep 10 22:40:23.494828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494776    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="89afae13-3808-408e-b64a-782597671e19" containerName="payload-pre-processing"
Sep 10 22:40:23.494828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494791    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e508224a-fcce-472b-a2cd-af03f0b71d04" containerName="payload-processing"
Sep 10 22:40:23.494828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494800    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e508224a-fcce-472b-a2cd-af03f0b71d04" containerName="payload-processing"
Sep 10 22:40:23.494828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494812    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4899ddae-25e1-4106-ad0c-e6d59abde486" containerName="revoke-keys"
Sep 10 22:40:23.494828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494821    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4899ddae-25e1-4106-ad0c-e6d59abde486" containerName="revoke-keys"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494836    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494844    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494855    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69e42a52-81e6-4a82-ae0a-6af1da7794b6" containerName="payload-processing"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494863    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="69e42a52-81e6-4a82-ae0a-6af1da7794b6" containerName="payload-processing"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494885    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31e4dbfc-5040-465c-a706-7c7905f6039c" containerName="main"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494893    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e4dbfc-5040-465c-a706-7c7905f6039c" containerName="main"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494905    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494913    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494921    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494929    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494937    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600fbe8c-42f5-4766-801d-d51c965e71b4" containerName="payload-processing"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494945    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="600fbe8c-42f5-4766-801d-d51c965e71b4" containerName="payload-processing"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494954    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" containerName="payload-pre-processing"
Sep 10 22:40:23.494994 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.494963    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" containerName="payload-pre-processing"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495041    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495053    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495065    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f0bb05-10f2-4a0c-b34a-97de6cc5e25c" containerName="payload-pre-processing"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495074    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495083    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="600fbe8c-42f5-4766-801d-d51c965e71b4" containerName="payload-processing"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495095    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c022a573-7bd3-4afb-bb5f-7da6e320fff7" containerName="istio-proxy"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495103    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="69e42a52-81e6-4a82-ae0a-6af1da7794b6" containerName="payload-processing"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495112    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4899ddae-25e1-4106-ad0c-e6d59abde486" containerName="revoke-keys"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495120    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="31e4dbfc-5040-465c-a706-7c7905f6039c" containerName="main"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495131    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e508224a-fcce-472b-a2cd-af03f0b71d04" containerName="payload-processing"
Sep 10 22:40:23.495410 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.495139    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="89afae13-3808-408e-b64a-782597671e19" containerName="payload-pre-processing"
Sep 10 22:40:23.498513 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.498494    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.500733 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.500713    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-512cd3c8-kserve-self-signed-certs\""
Sep 10 22:40:23.505530 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.505504    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"]
Sep 10 22:40:23.632857 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.632818    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-home\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.633121 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.632869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tls-certs\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.633121 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.632906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tjt4\" (UniqueName: \"kubernetes.io/projected/22945fb6-d05b-4c0a-b408-a7f735bc9a36-kube-api-access-9tjt4\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.633287 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.633121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tmp-dir\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.633349 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.633283    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-dshm\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.633428 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.633401    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-model-cache\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.734660 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.734624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tmp-dir\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.734708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-dshm\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.734733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-model-cache\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.734764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-home\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.734789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tls-certs\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.734927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tjt4\" (UniqueName: \"kubernetes.io/projected/22945fb6-d05b-4c0a-b408-a7f735bc9a36-kube-api-access-9tjt4\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735124 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.735079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tmp-dir\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.735134    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-model-cache\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.735391 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.735157    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-home\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.737261 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.737236    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-dshm\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.737387 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.737370    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tls-certs\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.742455 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.742420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tjt4\" (UniqueName: \"kubernetes.io/projected/22945fb6-d05b-4c0a-b408-a7f735bc9a36-kube-api-access-9tjt4\") pod \"e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") " pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:23.809328 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:23.809266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:24.077319 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:24.077291    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"]
Sep 10 22:40:24.080160 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:40:24.080133    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22945fb6_d05b_4c0a_b408_a7f735bc9a36.slice/crio-389a126af1316ad9cbbfda461351f349d27add06571e7d944556cfb04b52188a WatchSource:0}: Error finding container 389a126af1316ad9cbbfda461351f349d27add06571e7d944556cfb04b52188a: Status 404 returned error can't find the container with id 389a126af1316ad9cbbfda461351f349d27add06571e7d944556cfb04b52188a
Sep 10 22:40:24.424160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:24.424060    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k" event={"ID":"22945fb6-d05b-4c0a-b408-a7f735bc9a36","Type":"ContainerStarted","Data":"70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15"}
Sep 10 22:40:24.424160 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:24.424104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k" event={"ID":"22945fb6-d05b-4c0a-b408-a7f735bc9a36","Type":"ContainerStarted","Data":"389a126af1316ad9cbbfda461351f349d27add06571e7d944556cfb04b52188a"}
Sep 10 22:40:24.441432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:24.441375    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k" podStartSLOduration=1.441356689 podStartE2EDuration="1.441356689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:40:24.439085466 +0000 UTC m=+2093.823590891" watchObservedRunningTime="2026-09-10 22:40:24.441356689 +0000 UTC m=+2093.825862110"
Sep 10 22:40:31.225516 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:31.225481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:40:33.810105 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:33.810061    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:33.810522 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:33.810144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:33.822799 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:33.822772    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:34.481170 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:34.481142    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:40:37.418006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.417968    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"]
Sep 10 22:40:37.418494 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.418478    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy"
Sep 10 22:40:37.418562 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.418498    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="76c15550-c70e-4fce-a797-3cef528070db" containerName="istio-proxy"
Sep 10 22:40:37.421707 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.421685    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.424216 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.424183    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-512cd3c8-kserve-self-signed-certs\""
Sep 10 22:40:37.436006 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.435980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"]
Sep 10 22:40:37.438635 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.438611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73516783-5669-437f-9701-50de8f598ddb-tls-certs\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.438740 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.438650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-tmp-dir\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.438740 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.438676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm9dx\" (UniqueName: \"kubernetes.io/projected/73516783-5669-437f-9701-50de8f598ddb-kube-api-access-jm9dx\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.438740 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.438722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-home\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.438847 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.438780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-model-cache\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.438847 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.438801    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-dshm\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.539731 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.539693    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-tmp-dir\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.539908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.539745    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm9dx\" (UniqueName: \"kubernetes.io/projected/73516783-5669-437f-9701-50de8f598ddb-kube-api-access-jm9dx\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.539908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.539798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-home\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.539908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.539868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-model-cache\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.539908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.539902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-dshm\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.540122 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.539959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73516783-5669-437f-9701-50de8f598ddb-tls-certs\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.540122 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.540098    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-tmp-dir\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.540223 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.540206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-home\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.540277 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.540257    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-model-cache\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.542488 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.542466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-dshm\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.542844 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.542823    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73516783-5669-437f-9701-50de8f598ddb-tls-certs\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.547942 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.547918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm9dx\" (UniqueName: \"kubernetes.io/projected/73516783-5669-437f-9701-50de8f598ddb-kube-api-access-jm9dx\") pod \"e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") " pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:37.731218 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:37.731128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:38.010012 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:38.009981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"]
Sep 10 22:40:38.011736 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:40:38.011712    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73516783_5669_437f_9701_50de8f598ddb.slice/crio-dfd8cc3fed0dead8f1c81ff05baa34be5fafeb18f8514100b64031f931b0f8e7 WatchSource:0}: Error finding container dfd8cc3fed0dead8f1c81ff05baa34be5fafeb18f8514100b64031f931b0f8e7: Status 404 returned error can't find the container with id dfd8cc3fed0dead8f1c81ff05baa34be5fafeb18f8514100b64031f931b0f8e7
Sep 10 22:40:38.489648 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:38.489611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5" event={"ID":"73516783-5669-437f-9701-50de8f598ddb","Type":"ContainerStarted","Data":"e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285"}
Sep 10 22:40:38.490154 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:38.489659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5" event={"ID":"73516783-5669-437f-9701-50de8f598ddb","Type":"ContainerStarted","Data":"dfd8cc3fed0dead8f1c81ff05baa34be5fafeb18f8514100b64031f931b0f8e7"}
Sep 10 22:40:38.509817 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:38.509763    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5" podStartSLOduration=1.509742521 podStartE2EDuration="1.509742521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:40:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:40:38.507464041 +0000 UTC m=+2107.891969466" watchObservedRunningTime="2026-09-10 22:40:38.509742521 +0000 UTC m=+2107.894247947"
Sep 10 22:40:40.514672 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:40.514638    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:40:47.731914 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:47.731876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:47.732306 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:47.731935    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:47.744143 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:47.744115    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:40:48.548450 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:40:48.548402    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:41:05.656557 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:05.656523    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"]
Sep 10 22:41:06.895394 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:06.895362    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"]
Sep 10 22:41:08.640272 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:08.640239    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:41:15.723784 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:15.723746    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:41:20.675631 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:20.675583    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k" podUID="22945fb6-d05b-4c0a-b408-a7f735bc9a36" containerName="main" containerID="cri-o://70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15" gracePeriod=45
Sep 10 22:41:20.918070 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:20.918045    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:41:21.014163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014129    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-dshm\") pod \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") "
Sep 10 22:41:21.014356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014193    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-home\") pod \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") "
Sep 10 22:41:21.014356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014243    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tls-certs\") pod \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") "
Sep 10 22:41:21.014356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014300    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-model-cache\") pod \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") "
Sep 10 22:41:21.014356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014324    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tjt4\" (UniqueName: \"kubernetes.io/projected/22945fb6-d05b-4c0a-b408-a7f735bc9a36-kube-api-access-9tjt4\") pod \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") "
Sep 10 22:41:21.014356 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014345    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tmp-dir\") pod \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\" (UID: \"22945fb6-d05b-4c0a-b408-a7f735bc9a36\") "
Sep 10 22:41:21.014647 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014526    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-home" (OuterVolumeSpecName: "home") pod "22945fb6-d05b-4c0a-b408-a7f735bc9a36" (UID: "22945fb6-d05b-4c0a-b408-a7f735bc9a36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:21.014647 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014560    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-model-cache" (OuterVolumeSpecName: "model-cache") pod "22945fb6-d05b-4c0a-b408-a7f735bc9a36" (UID: "22945fb6-d05b-4c0a-b408-a7f735bc9a36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:21.014726 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.014688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22945fb6-d05b-4c0a-b408-a7f735bc9a36" (UID: "22945fb6-d05b-4c0a-b408-a7f735bc9a36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:21.016387 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.016362    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-dshm" (OuterVolumeSpecName: "dshm") pod "22945fb6-d05b-4c0a-b408-a7f735bc9a36" (UID: "22945fb6-d05b-4c0a-b408-a7f735bc9a36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:21.016590 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.016560    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22945fb6-d05b-4c0a-b408-a7f735bc9a36" (UID: "22945fb6-d05b-4c0a-b408-a7f735bc9a36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:41:21.016679 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.016655    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22945fb6-d05b-4c0a-b408-a7f735bc9a36-kube-api-access-9tjt4" (OuterVolumeSpecName: "kube-api-access-9tjt4") pod "22945fb6-d05b-4c0a-b408-a7f735bc9a36" (UID: "22945fb6-d05b-4c0a-b408-a7f735bc9a36"). InnerVolumeSpecName "kube-api-access-9tjt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:41:21.115309 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.115272    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:21.115309 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.115299    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:21.115309 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.115308    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:21.115564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.115317    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22945fb6-d05b-4c0a-b408-a7f735bc9a36-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:21.115564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.115330    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22945fb6-d05b-4c0a-b408-a7f735bc9a36-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:21.115564 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.115339    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tjt4\" (UniqueName: \"kubernetes.io/projected/22945fb6-d05b-4c0a-b408-a7f735bc9a36-kube-api-access-9tjt4\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:21.686670 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.686630    2563 generic.go:356] "Generic (PLEG): container finished" podID="22945fb6-d05b-4c0a-b408-a7f735bc9a36" containerID="70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15" exitCode=0
Sep 10 22:41:21.687097 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.686695    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"
Sep 10 22:41:21.687097 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.686719    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k" event={"ID":"22945fb6-d05b-4c0a-b408-a7f735bc9a36","Type":"ContainerDied","Data":"70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15"}
Sep 10 22:41:21.687097 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.686769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k" event={"ID":"22945fb6-d05b-4c0a-b408-a7f735bc9a36","Type":"ContainerDied","Data":"389a126af1316ad9cbbfda461351f349d27add06571e7d944556cfb04b52188a"}
Sep 10 22:41:21.687097 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.686786    2563 scope.go:117] "RemoveContainer" containerID="70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15"
Sep 10 22:41:21.695432 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.695414    2563 scope.go:117] "RemoveContainer" containerID="70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15"
Sep 10 22:41:21.695743 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:41:21.695724    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15\": container with ID starting with 70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15 not found: ID does not exist" containerID="70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15"
Sep 10 22:41:21.695790 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.695755    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15"} err="failed to get container status \"70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15\": rpc error: code = NotFound desc = could not find container \"70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15\": container with ID starting with 70e7ed58fa6d21838a8a142c51328ab5676552aa8a623da81ce8166743885f15 not found: ID does not exist"
Sep 10 22:41:21.703239 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.703218    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"]
Sep 10 22:41:21.708889 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.708869    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-512cd3c8-kserve-5ff8fd6ddc-sd77k"]
Sep 10 22:41:21.915166 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:21.915057    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5" podUID="73516783-5669-437f-9701-50de8f598ddb" containerName="main" containerID="cri-o://e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285" gracePeriod=45
Sep 10 22:41:22.162764 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.162740    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:41:22.326222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326146    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-dshm\") pod \"73516783-5669-437f-9701-50de8f598ddb\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") "
Sep 10 22:41:22.326222 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326194    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-home\") pod \"73516783-5669-437f-9701-50de8f598ddb\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") "
Sep 10 22:41:22.326415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326256    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-tmp-dir\") pod \"73516783-5669-437f-9701-50de8f598ddb\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") "
Sep 10 22:41:22.326415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326302    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73516783-5669-437f-9701-50de8f598ddb-tls-certs\") pod \"73516783-5669-437f-9701-50de8f598ddb\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") "
Sep 10 22:41:22.326415 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326349    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm9dx\" (UniqueName: \"kubernetes.io/projected/73516783-5669-437f-9701-50de8f598ddb-kube-api-access-jm9dx\") pod \"73516783-5669-437f-9701-50de8f598ddb\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") "
Sep 10 22:41:22.326614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326427    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-model-cache\") pod \"73516783-5669-437f-9701-50de8f598ddb\" (UID: \"73516783-5669-437f-9701-50de8f598ddb\") "
Sep 10 22:41:22.326614 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326540    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-home" (OuterVolumeSpecName: "home") pod "73516783-5669-437f-9701-50de8f598ddb" (UID: "73516783-5669-437f-9701-50de8f598ddb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:22.326697 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326604    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "73516783-5669-437f-9701-50de8f598ddb" (UID: "73516783-5669-437f-9701-50de8f598ddb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:22.326774 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326751    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-model-cache" (OuterVolumeSpecName: "model-cache") pod "73516783-5669-437f-9701-50de8f598ddb" (UID: "73516783-5669-437f-9701-50de8f598ddb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:22.326840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326809    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-tmp-dir\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:22.326840 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.326826    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-home\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:22.328527 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.328498    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-dshm" (OuterVolumeSpecName: "dshm") pod "73516783-5669-437f-9701-50de8f598ddb" (UID: "73516783-5669-437f-9701-50de8f598ddb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 22:41:22.328647 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.328536    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73516783-5669-437f-9701-50de8f598ddb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "73516783-5669-437f-9701-50de8f598ddb" (UID: "73516783-5669-437f-9701-50de8f598ddb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:41:22.328647 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.328566    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73516783-5669-437f-9701-50de8f598ddb-kube-api-access-jm9dx" (OuterVolumeSpecName: "kube-api-access-jm9dx") pod "73516783-5669-437f-9701-50de8f598ddb" (UID: "73516783-5669-437f-9701-50de8f598ddb"). InnerVolumeSpecName "kube-api-access-jm9dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:41:22.427573 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.427539    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73516783-5669-437f-9701-50de8f598ddb-tls-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:22.427573 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.427570    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm9dx\" (UniqueName: \"kubernetes.io/projected/73516783-5669-437f-9701-50de8f598ddb-kube-api-access-jm9dx\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:22.427573 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.427580    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-model-cache\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:22.427763 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.427588    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73516783-5669-437f-9701-50de8f598ddb-dshm\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:22.692929 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.692840    2563 generic.go:356] "Generic (PLEG): container finished" podID="73516783-5669-437f-9701-50de8f598ddb" containerID="e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285" exitCode=0
Sep 10 22:41:22.692929 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.692881    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"
Sep 10 22:41:22.693479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.692925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5" event={"ID":"73516783-5669-437f-9701-50de8f598ddb","Type":"ContainerDied","Data":"e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285"}
Sep 10 22:41:22.693479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.692964    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5" event={"ID":"73516783-5669-437f-9701-50de8f598ddb","Type":"ContainerDied","Data":"dfd8cc3fed0dead8f1c81ff05baa34be5fafeb18f8514100b64031f931b0f8e7"}
Sep 10 22:41:22.693479 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.692987    2563 scope.go:117] "RemoveContainer" containerID="e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285"
Sep 10 22:41:22.702081 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.702061    2563 scope.go:117] "RemoveContainer" containerID="e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285"
Sep 10 22:41:22.702405 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:41:22.702387    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285\": container with ID starting with e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285 not found: ID does not exist" containerID="e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285"
Sep 10 22:41:22.702506 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.702415    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285"} err="failed to get container status \"e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285\": rpc error: code = NotFound desc = could not find container \"e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285\": container with ID starting with e071503f678db47fc3c227af59e9b1b64a1eed935dc1d6095d7196e8cf877285 not found: ID does not exist"
Sep 10 22:41:22.714795 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.714754    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"]
Sep 10 22:41:22.721419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:22.721390    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-512cd3c8-kserve-7f7f46f68c-zxvm5"]
Sep 10 22:41:23.110018 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.109990    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22945fb6-d05b-4c0a-b408-a7f735bc9a36" path="/var/lib/kubelet/pods/22945fb6-d05b-4c0a-b408-a7f735bc9a36/volumes"
Sep 10 22:41:23.110394 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.110381    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73516783-5669-437f-9701-50de8f598ddb" path="/var/lib/kubelet/pods/73516783-5669-437f-9701-50de8f598ddb/volumes"
Sep 10 22:41:23.507048 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.507012    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:41:23.913800 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.913718    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-hwt8z"]
Sep 10 22:41:23.914163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.914039    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73516783-5669-437f-9701-50de8f598ddb" containerName="main"
Sep 10 22:41:23.914163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.914050    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="73516783-5669-437f-9701-50de8f598ddb" containerName="main"
Sep 10 22:41:23.914163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.914062    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22945fb6-d05b-4c0a-b408-a7f735bc9a36" containerName="main"
Sep 10 22:41:23.914163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.914067    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="22945fb6-d05b-4c0a-b408-a7f735bc9a36" containerName="main"
Sep 10 22:41:23.914163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.914125    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="73516783-5669-437f-9701-50de8f598ddb" containerName="main"
Sep 10 22:41:23.914163 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.914135    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="22945fb6-d05b-4c0a-b408-a7f735bc9a36" containerName="main"
Sep 10 22:41:23.918361 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.918344    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:23.921530 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.921508    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 22:41:23.921530 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.921518    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zvqzg\""
Sep 10 22:41:23.921713 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.921554    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 22:41:23.921713 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.921516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 22:41:23.921713 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.921670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 22:41:23.930463 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:23.930426    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-hwt8z"]
Sep 10 22:41:24.041747 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.041714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-metrics-certs\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.041915 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.041800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntltf\" (UniqueName: \"kubernetes.io/projected/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-kube-api-access-ntltf\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.041915 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.041822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-openshift-service-ca\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.041915 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.041849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-maas-api-tls\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.142733 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.142705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-maas-api-tls\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.142880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.142758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-metrics-certs\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.142880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.142817    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntltf\" (UniqueName: \"kubernetes.io/projected/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-kube-api-access-ntltf\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.142880 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.142841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-openshift-service-ca\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.143684 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.143642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-openshift-service-ca\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.145651 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.145622    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-maas-api-tls\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.145746 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.145686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-metrics-certs\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.151061 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.151038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntltf\" (UniqueName: \"kubernetes.io/projected/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-kube-api-access-ntltf\") pod \"maas-api-554988677c-hwt8z\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") " pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.228742 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.228663    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:24.498942 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.498919    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-hwt8z"]
Sep 10 22:41:24.500903 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:41:24.500872    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35d2e9f_2355_4b75_869c_b34e9d2f7b79.slice/crio-7f7f2ae503fe08079e73200886f480ff036159c3a921c4ad2e49798fcb1d8c60 WatchSource:0}: Error finding container 7f7f2ae503fe08079e73200886f480ff036159c3a921c4ad2e49798fcb1d8c60: Status 404 returned error can't find the container with id 7f7f2ae503fe08079e73200886f480ff036159c3a921c4ad2e49798fcb1d8c60
Sep 10 22:41:24.502419 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.502396    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 22:41:24.705406 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:24.705363    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z" event={"ID":"a35d2e9f-2355-4b75-869c-b34e9d2f7b79","Type":"ContainerStarted","Data":"7f7f2ae503fe08079e73200886f480ff036159c3a921c4ad2e49798fcb1d8c60"}
Sep 10 22:41:27.720321 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:27.720280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z" event={"ID":"a35d2e9f-2355-4b75-869c-b34e9d2f7b79","Type":"ContainerStarted","Data":"9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304"}
Sep 10 22:41:27.720756 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:27.720363    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:27.737922 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:27.737873    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z" podStartSLOduration=2.419742328 podStartE2EDuration="4.737858521s" totalImagesPullingTime="2.318116193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:41:27.737362199 +0000 UTC m=+2157.121867626" watchObservedRunningTime="2026-09-10 22:41:27.737858521 +0000 UTC m=+2157.122363945"
Sep 10 22:41:31.406972 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:31.406932    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:41:31.882540 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:31.882508    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-hwt8z"]
Sep 10 22:41:31.882763 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:31.882726    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z" podUID="a35d2e9f-2355-4b75-869c-b34e9d2f7b79" containerName="maas-api" containerID="cri-o://9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304" gracePeriod=30
Sep 10 22:41:31.888287 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:31.888265    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:32.130700 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.130674    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:32.211151 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.211071    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntltf\" (UniqueName: \"kubernetes.io/projected/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-kube-api-access-ntltf\") pod \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") "
Sep 10 22:41:32.211151 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.211106    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-openshift-service-ca\") pod \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") "
Sep 10 22:41:32.211385 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.211161    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-metrics-certs\") pod \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") "
Sep 10 22:41:32.211385 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.211205    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-maas-api-tls\") pod \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\" (UID: \"a35d2e9f-2355-4b75-869c-b34e9d2f7b79\") "
Sep 10 22:41:32.211561 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.211532    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a35d2e9f-2355-4b75-869c-b34e9d2f7b79" (UID: "a35d2e9f-2355-4b75-869c-b34e9d2f7b79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 22:41:32.213499 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.213471    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a35d2e9f-2355-4b75-869c-b34e9d2f7b79" (UID: "a35d2e9f-2355-4b75-869c-b34e9d2f7b79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:41:32.213611 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.213536    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a35d2e9f-2355-4b75-869c-b34e9d2f7b79" (UID: "a35d2e9f-2355-4b75-869c-b34e9d2f7b79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 22:41:32.213844 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.213815    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-kube-api-access-ntltf" (OuterVolumeSpecName: "kube-api-access-ntltf") pod "a35d2e9f-2355-4b75-869c-b34e9d2f7b79" (UID: "a35d2e9f-2355-4b75-869c-b34e9d2f7b79"). InnerVolumeSpecName "kube-api-access-ntltf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 22:41:32.311828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.311798    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-maas-api-tls\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:32.311828 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.311827    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntltf\" (UniqueName: \"kubernetes.io/projected/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-kube-api-access-ntltf\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:32.312055 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.311839    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-openshift-service-ca\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:32.312055 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.311852    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a35d2e9f-2355-4b75-869c-b34e9d2f7b79-metrics-certs\") on node \"ip-10-0-137-25.ec2.internal\" DevicePath \"\""
Sep 10 22:41:32.744654 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.744618    2563 generic.go:356] "Generic (PLEG): container finished" podID="a35d2e9f-2355-4b75-869c-b34e9d2f7b79" containerID="9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304" exitCode=0
Sep 10 22:41:32.745137 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.744665    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z" event={"ID":"a35d2e9f-2355-4b75-869c-b34e9d2f7b79","Type":"ContainerDied","Data":"9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304"}
Sep 10 22:41:32.745137 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.744667    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z"
Sep 10 22:41:32.745137 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.744708    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-554988677c-hwt8z" event={"ID":"a35d2e9f-2355-4b75-869c-b34e9d2f7b79","Type":"ContainerDied","Data":"7f7f2ae503fe08079e73200886f480ff036159c3a921c4ad2e49798fcb1d8c60"}
Sep 10 22:41:32.745137 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.744730    2563 scope.go:117] "RemoveContainer" containerID="9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304"
Sep 10 22:41:32.754063 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.754045    2563 scope.go:117] "RemoveContainer" containerID="9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304"
Sep 10 22:41:32.754345 ip-10-0-137-25 kubenswrapper[2563]: E0910 22:41:32.754326    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304\": container with ID starting with 9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304 not found: ID does not exist" containerID="9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304"
Sep 10 22:41:32.754392 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.754353    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304"} err="failed to get container status \"9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304\": rpc error: code = NotFound desc = could not find container \"9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304\": container with ID starting with 9a4f4f5c89663b8b3bca7f4a3788dba3e4620415b95e17781508b5a002cdb304 not found: ID does not exist"
Sep 10 22:41:32.765422 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.765397    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-hwt8z"]
Sep 10 22:41:32.774078 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:32.774054    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-554988677c-hwt8z"]
Sep 10 22:41:33.111131 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:33.111053    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a35d2e9f-2355-4b75-869c-b34e9d2f7b79" path="/var/lib/kubelet/pods/a35d2e9f-2355-4b75-869c-b34e9d2f7b79/volumes"
Sep 10 22:41:38.012839 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:38.012807    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:41:43.813588 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:43.813555    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:41:50.812054 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:50.812019    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-d7djs"]
Sep 10 22:41:59.502581 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:41:59.502546    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-b7xh5_dfb1b1da-5571-4944-a033-8a40166d518a/manager/0.log"
Sep 10 22:42:00.251385 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:00.251354    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-995c55587-gx5g8_14da3c6f-0401-40ec-b7b4-3371607fc5ff/copy-manifests/0.log"
Sep 10 22:42:00.258868 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:00.258844    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-995c55587-gx5g8_14da3c6f-0401-40ec-b7b4-3371607fc5ff/manager/0.log"
Sep 10 22:42:01.990322 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:01.990288    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-dqts2_2e862025-2635-45ab-b18e-f5cbab619dc1/manager/0.log"
Sep 10 22:42:02.096083 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:02.096054    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-l4tlz_ad91562f-dc92-4593-908e-1e6119f2442c/manager/0.log"
Sep 10 22:42:02.210655 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:02.210626    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-s7kqv_0e813661-5255-4c9f-b12e-29c56d887527/kuadrant-console-plugin/0.log"
Sep 10 22:42:02.438749 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:02.438676    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-d7djs_502c096c-f9b7-4031-9f45-c8ab5f467fd0/limitador/0.log"
Sep 10 22:42:02.553057 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:02.553029    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-hx57f_1633881a-0359-42f7-ab23-37d1dbaa9fea/manager/0.log"
Sep 10 22:42:02.892517 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:02.892488    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj_659c8c68-059b-41b0-ad38-1f4b441fd841/istio-proxy/0.log"
Sep 10 22:42:03.224304 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:03.224229    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b867fd64-rsmwb_971ec54a-9d87-48ee-9465-b7f3d685822c/kube-auth-proxy/0.log"
Sep 10 22:42:03.336861 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:03.336829    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-2784q_1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec/istio-proxy/1.log"
Sep 10 22:42:03.461099 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:03.461067    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-d88hf_9e75642a-f92b-40cf-8223-0902432b07f5/payload-pre-processing/0.log"
Sep 10 22:42:04.034648 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:04.034620    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-2p2z6_81e57b7c-fa1f-4330-9ae3-ce1676d51530/main/0.log"
Sep 10 22:42:04.376941 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:04.376864    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-66h7v_ffad8d35-0731-4da0-8bc7-451d5f8a2ff5/main/0.log"
Sep 10 22:42:04.728134 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:04.728066    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gxcm7_4cc432e1-0fe2-44a4-93ce-a507a67751a7/main/0.log"
Sep 10 22:42:10.518398 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.518363    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f4kkc/must-gather-b9w54"]
Sep 10 22:42:10.518856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.518727    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a35d2e9f-2355-4b75-869c-b34e9d2f7b79" containerName="maas-api"
Sep 10 22:42:10.518856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.518740    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a35d2e9f-2355-4b75-869c-b34e9d2f7b79" containerName="maas-api"
Sep 10 22:42:10.518856 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.518801    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a35d2e9f-2355-4b75-869c-b34e9d2f7b79" containerName="maas-api"
Sep 10 22:42:10.521915 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.521894    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:10.524123 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.524103    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f4kkc\"/\"openshift-service-ca.crt\""
Sep 10 22:42:10.524226 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.524150    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f4kkc\"/\"default-dockercfg-8m4xn\""
Sep 10 22:42:10.524806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.524790    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f4kkc\"/\"kube-root-ca.crt\""
Sep 10 22:42:10.532949 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.532927    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f4kkc/must-gather-b9w54"]
Sep 10 22:42:10.617014 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.616981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms2nl\" (UniqueName: \"kubernetes.io/projected/9d74fd9b-0179-48ad-bf87-e6afcbd67a93-kube-api-access-ms2nl\") pod \"must-gather-b9w54\" (UID: \"9d74fd9b-0179-48ad-bf87-e6afcbd67a93\") " pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:10.617165 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.617028    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d74fd9b-0179-48ad-bf87-e6afcbd67a93-must-gather-output\") pod \"must-gather-b9w54\" (UID: \"9d74fd9b-0179-48ad-bf87-e6afcbd67a93\") " pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:10.718502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.718463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms2nl\" (UniqueName: \"kubernetes.io/projected/9d74fd9b-0179-48ad-bf87-e6afcbd67a93-kube-api-access-ms2nl\") pod \"must-gather-b9w54\" (UID: \"9d74fd9b-0179-48ad-bf87-e6afcbd67a93\") " pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:10.718651 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.718525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d74fd9b-0179-48ad-bf87-e6afcbd67a93-must-gather-output\") pod \"must-gather-b9w54\" (UID: \"9d74fd9b-0179-48ad-bf87-e6afcbd67a93\") " pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:10.718806 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.718791    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d74fd9b-0179-48ad-bf87-e6afcbd67a93-must-gather-output\") pod \"must-gather-b9w54\" (UID: \"9d74fd9b-0179-48ad-bf87-e6afcbd67a93\") " pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:10.731786 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.731754    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms2nl\" (UniqueName: \"kubernetes.io/projected/9d74fd9b-0179-48ad-bf87-e6afcbd67a93-kube-api-access-ms2nl\") pod \"must-gather-b9w54\" (UID: \"9d74fd9b-0179-48ad-bf87-e6afcbd67a93\") " pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:10.830757 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:10.830670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f4kkc/must-gather-b9w54"
Sep 10 22:42:11.100177 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:11.100151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f4kkc/must-gather-b9w54"]
Sep 10 22:42:11.102717 ip-10-0-137-25 kubenswrapper[2563]: W0910 22:42:11.102693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d74fd9b_0179_48ad_bf87_e6afcbd67a93.slice/crio-7458eb4660d554be9acf7b973aed2d72d25241d3121a39e92b533df1dbaa9969 WatchSource:0}: Error finding container 7458eb4660d554be9acf7b973aed2d72d25241d3121a39e92b533df1dbaa9969: Status 404 returned error can't find the container with id 7458eb4660d554be9acf7b973aed2d72d25241d3121a39e92b533df1dbaa9969
Sep 10 22:42:11.927100 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:11.927075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f4kkc/must-gather-b9w54" event={"ID":"9d74fd9b-0179-48ad-bf87-e6afcbd67a93","Type":"ContainerStarted","Data":"f97d552757305fd7a5d7496d13dc3f83880390229133cec43549d680bbd75034"}
Sep 10 22:42:11.927413 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:11.927114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f4kkc/must-gather-b9w54" event={"ID":"9d74fd9b-0179-48ad-bf87-e6afcbd67a93","Type":"ContainerStarted","Data":"7458eb4660d554be9acf7b973aed2d72d25241d3121a39e92b533df1dbaa9969"}
Sep 10 22:42:12.938986 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:12.938369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f4kkc/must-gather-b9w54" event={"ID":"9d74fd9b-0179-48ad-bf87-e6afcbd67a93","Type":"ContainerStarted","Data":"c6df0ba21bc4f76193ea8ae75956abf0fcca9458e1bad012299ec1f7e6c7f34b"}
Sep 10 22:42:12.954534 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:12.954478    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f4kkc/must-gather-b9w54" podStartSLOduration=2.228584573 podStartE2EDuration="2.954429504s" totalImagesPullingTime="725.844931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 22:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:42:12.953834528 +0000 UTC m=+2202.338339955" watchObservedRunningTime="2026-09-10 22:42:12.954429504 +0000 UTC m=+2202.338934931"
Sep 10 22:42:13.353496 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:13.353470    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k8cm5_f5e33414-abf8-4ebc-a2d2-22d60d529098/global-pull-secret-syncer/0.log"
Sep 10 22:42:13.574816 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:13.574788    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dz966_8b115e6d-22b2-422a-8e9a-2ab56b60e746/konnectivity-agent/0.log"
Sep 10 22:42:13.626877 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:13.626805    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-25.ec2.internal_f9996d37233f3f53259991758723d858/haproxy/0.log"
Sep 10 22:42:18.734980 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:18.734946    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-dqts2_2e862025-2635-45ab-b18e-f5cbab619dc1/manager/0.log"
Sep 10 22:42:18.807202 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:18.807172    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-l4tlz_ad91562f-dc92-4593-908e-1e6119f2442c/manager/0.log"
Sep 10 22:42:18.836071 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:18.836040    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-s7kqv_0e813661-5255-4c9f-b12e-29c56d887527/kuadrant-console-plugin/0.log"
Sep 10 22:42:18.915032 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:18.915004    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-d7djs_502c096c-f9b7-4031-9f45-c8ab5f467fd0/limitador/0.log"
Sep 10 22:42:18.988802 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:18.988712    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-hx57f_1633881a-0359-42f7-ab23-37d1dbaa9fea/manager/0.log"
Sep 10 22:42:20.857765 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:20.857736    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b86j5_dab3fe50-4a30-4571-ab66-fc8463d9276d/node-exporter/0.log"
Sep 10 22:42:20.885765 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:20.885737    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b86j5_dab3fe50-4a30-4571-ab66-fc8463d9276d/kube-rbac-proxy/0.log"
Sep 10 22:42:20.909604 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:20.909581    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b86j5_dab3fe50-4a30-4571-ab66-fc8463d9276d/init-textfile/0.log"
Sep 10 22:42:23.047527 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.047487    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"]
Sep 10 22:42:23.052546 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.052518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.062003 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.061775    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"]
Sep 10 22:42:23.156618 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.156575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-proc\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.156835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.156656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-lib-modules\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.156835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.156688    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gljx8\" (UniqueName: \"kubernetes.io/projected/b542c82f-de94-43cb-85ed-0788830b1ad5-kube-api-access-gljx8\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.156835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.156717    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-sys\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.156835 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.156788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-podres\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.258038 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.257997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-sys\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.258502 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.258478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-podres\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.258853 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.258833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-proc\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.259117 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.259100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-lib-modules\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.259261 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.259244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gljx8\" (UniqueName: \"kubernetes.io/projected/b542c82f-de94-43cb-85ed-0788830b1ad5-kube-api-access-gljx8\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.259397 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.258732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-podres\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.259535 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.258350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-sys\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.259637 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.259301    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-lib-modules\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.259726 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.259013    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b542c82f-de94-43cb-85ed-0788830b1ad5-proc\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.269011 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.268974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gljx8\" (UniqueName: \"kubernetes.io/projected/b542c82f-de94-43cb-85ed-0788830b1ad5-kube-api-access-gljx8\") pod \"perf-node-gather-daemonset-kpz56\" (UID: \"b542c82f-de94-43cb-85ed-0788830b1ad5\") " pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.366945 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.366864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:23.654902 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:23.654808    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"]
Sep 10 22:42:24.033910 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:24.033868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56" event={"ID":"b542c82f-de94-43cb-85ed-0788830b1ad5","Type":"ContainerStarted","Data":"f43b38640a2d684c7b4e216fdff3f17f8d231b087c9ec8ed9e85a26b20fca047"}
Sep 10 22:42:24.034138 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:24.033941    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56" event={"ID":"b542c82f-de94-43cb-85ed-0788830b1ad5","Type":"ContainerStarted","Data":"f592a435f345ce81c7db8ffb5a7a1f0ed389095245160ef6b871e0eeeaf8953a"}
Sep 10 22:42:24.034221 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:24.034173    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:24.052664 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:24.052614    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56" podStartSLOduration=1.052596284 podStartE2EDuration="1.052596284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 22:42:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 22:42:24.050429832 +0000 UTC m=+2213.434935257" watchObservedRunningTime="2026-09-10 22:42:24.052596284 +0000 UTC m=+2213.437101709"
Sep 10 22:42:25.852497 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:25.852465    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mxg8z_70421c90-6fde-44da-aa1d-cf6b7a01f6dc/dns/0.log"
Sep 10 22:42:25.873841 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:25.873813    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mxg8z_70421c90-6fde-44da-aa1d-cf6b7a01f6dc/kube-rbac-proxy/0.log"
Sep 10 22:42:25.895882 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:25.895853    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-77qcj_c58b53b6-ef61-400e-b3b4-beaf17da2443/dns-node-resolver/0.log"
Sep 10 22:42:26.531088 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:26.531060    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vk5zq_9ddc1a73-dcad-4177-ae60-ed096db3624d/node-ca/0.log"
Sep 10 22:42:27.468588 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:27.468548    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj_659c8c68-059b-41b0-ad38-1f4b441fd841/istio-proxy/0.log"
Sep 10 22:42:27.724187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:27.724096    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b867fd64-rsmwb_971ec54a-9d87-48ee-9465-b7f3d685822c/kube-auth-proxy/0.log"
Sep 10 22:42:27.756837 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:27.756807    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-2784q_1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec/istio-proxy/0.log"
Sep 10 22:42:27.769187 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:27.769154    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-2784q_1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec/istio-proxy/1.log"
Sep 10 22:42:27.805676 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:27.805645    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-d88hf_9e75642a-f92b-40cf-8223-0902432b07f5/payload-pre-processing/0.log"
Sep 10 22:42:28.729759 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:28.729724    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2k4zb_c6f77c79-5458-4700-81bb-ea60234efad2/serve-healthcheck-canary/0.log"
Sep 10 22:42:29.409503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:29.409463    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pplx7_3e3d07f6-baf1-45e6-919c-12bf63bf28ff/kube-rbac-proxy/0.log"
Sep 10 22:42:29.428582 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:29.428553    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pplx7_3e3d07f6-baf1-45e6-919c-12bf63bf28ff/exporter/0.log"
Sep 10 22:42:29.449683 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:29.449641    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pplx7_3e3d07f6-baf1-45e6-919c-12bf63bf28ff/extractor/0.log"
Sep 10 22:42:30.051672 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:30.051646    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f4kkc/perf-node-gather-daemonset-kpz56"
Sep 10 22:42:31.823350 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:31.823242    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-b7xh5_dfb1b1da-5571-4944-a033-8a40166d518a/manager/0.log"
Sep 10 22:42:32.160089 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:32.160053    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-995c55587-gx5g8_14da3c6f-0401-40ec-b7b4-3371607fc5ff/manager/0.log"
Sep 10 22:42:32.184908 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:32.184874    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-995c55587-gx5g8_14da3c6f-0401-40ec-b7b4-3371607fc5ff/copy-manifests/0.log"
Sep 10 22:42:33.707877 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:33.707848    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pmjpv_f1c52885-2908-449a-adca-6ff32128f87b/openshift-lws-operator/0.log"
Sep 10 22:42:33.805034 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:33.804998    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj_659c8c68-059b-41b0-ad38-1f4b441fd841/istio-proxy/0.log"
Sep 10 22:42:34.044808 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:34.044776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b867fd64-rsmwb_971ec54a-9d87-48ee-9465-b7f3d685822c/kube-auth-proxy/0.log"
Sep 10 22:42:34.064851 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:34.064792    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-2784q_1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec/istio-proxy/0.log"
Sep 10 22:42:34.074246 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:34.074217    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-2784q_1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec/istio-proxy/1.log"
Sep 10 22:42:34.104261 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:34.104236    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-d88hf_9e75642a-f92b-40cf-8223-0902432b07f5/payload-pre-processing/0.log"
Sep 10 22:42:35.643777 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:35.643749    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799c6g5lj_659c8c68-059b-41b0-ad38-1f4b441fd841/istio-proxy/0.log"
Sep 10 22:42:35.888204 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:35.888172    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9b867fd64-rsmwb_971ec54a-9d87-48ee-9465-b7f3d685822c/kube-auth-proxy/0.log"
Sep 10 22:42:35.916162 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:35.916070    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-2784q_1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec/istio-proxy/0.log"
Sep 10 22:42:35.926293 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:35.926269    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-2784q_1c8f1cb2-bf12-48a1-a2ab-dcca5b0d53ec/istio-proxy/1.log"
Sep 10 22:42:35.956696 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:35.956670    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-d88hf_9e75642a-f92b-40cf-8223-0902432b07f5/payload-pre-processing/0.log"
Sep 10 22:42:38.668503 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:38.668477    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-rt78z_39502c8f-7404-4c79-87a2-bbe3bf9ab7bf/migrator/0.log"
Sep 10 22:42:38.697512 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:38.697488    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-rt78z_39502c8f-7404-4c79-87a2-bbe3bf9ab7bf/graceful-termination/0.log"
Sep 10 22:42:40.182287 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:40.182259    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4v5g8_cb4095b3-21dd-4367-af3e-0aced8f43716/kube-multus-additional-cni-plugins/0.log"
Sep 10 22:42:40.204629 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:40.204600    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4v5g8_cb4095b3-21dd-4367-af3e-0aced8f43716/egress-router-binary-copy/0.log"
Sep 10 22:42:40.228591 ip-10-0-137-25 kubenswrapper[2563]: I0910 22:42:40.228542    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4v5g8_cb4095b3-21dd-4367-af3e-0aced8f43716/cni-plugins/0.log"