Sep 10 13:15:04.097111 ip-10-0-137-9 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 13:15:04.097126 ip-10-0-137-9 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 13:15:04.097136 ip-10-0-137-9 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 13:15:04.097355 ip-10-0-137-9 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 13:15:14.252527 ip-10-0-137-9 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 13:15:14.252543 ip-10-0-137-9 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 02ef6d5b34e6497887739e2b51e3a9cc --
Sep 10 13:17:14.690984 ip-10-0-137-9 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 13:17:15.036838 ip-10-0-137-9 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:17:15.036838 ip-10-0-137-9 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 13:17:15.036838 ip-10-0-137-9 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:17:15.036838 ip-10-0-137-9 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 13:17:15.036838 ip-10-0-137-9 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 13:17:15.037566 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.037494    2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 13:17:15.041044 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041030    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041045    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041049    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041052    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041056    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041059    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041062    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041065    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041068    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041071    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041075    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041078    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041081    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041084    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:17:15.041082 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041089    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041093    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041096    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041099    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041102    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041105    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041108    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041110    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041113    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041116    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041120    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041123    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041126    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041130    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041132    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041135    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041138    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041141    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041144    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:17:15.041419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041146    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041149    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041152    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041154    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041157    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041159    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041162    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041164    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041167    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041170    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041172    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041175    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041177    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041180    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041182    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041186    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041189    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041192    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041194    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041197    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:17:15.041896 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041200    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041203    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041207    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041209    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041212    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041215    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041217    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041220    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041223    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041225    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041228    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041230    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041233    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041235    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041238    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041240    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041243    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041246    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041249    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041251    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:17:15.042377 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041254    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041256    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041259    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041261    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041264    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041266    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041269    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041272    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041275    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041278    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041281    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041284    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041287    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041290    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041292    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041296    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041298    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041301    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041304    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041306    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:17:15.042917 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041309    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041311    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041314    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041316    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041319    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.041322    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042056    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042064    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042067    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042070    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042074    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042077    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042080    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042083    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042086    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042089    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042091    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042094    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042097    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042102    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:17:15.043427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042105    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042107    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042110    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042113    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042115    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042118    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042121    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042124    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042126    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042129    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042131    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042134    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042136    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042139    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042141    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042144    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042146    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042149    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042152    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042154    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:17:15.043933 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042157    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042159    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042162    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042164    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042167    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042170    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042173    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042175    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042178    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042180    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042183    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042186    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042189    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042192    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042194    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042197    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042199    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042202    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042205    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042207    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:17:15.044430 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042210    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042212    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042215    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042217    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042220    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042223    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042226    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042228    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042231    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042234    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042236    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042239    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042241    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042244    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042247    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042249    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042252    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042255    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042257    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:17:15.044932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042261    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042264    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042266    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042269    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042272    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042275    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042279    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042281    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042284    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042286    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042289    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042291    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042294    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042296    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042299    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042301    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042304    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042307    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042309    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042312    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:17:15.045415 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042314    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042317    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042319    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042323    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042326    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042330    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042727    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042733    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042736    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042739    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042743    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042746    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042749    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042752    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042754    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042757    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042761    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042763    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042766    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042769    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:17:15.045925 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042771    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042774    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042777    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042780    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042782    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042785    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042788    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042791    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042793    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042796    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042800    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042803    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042806    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042808    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042811    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042813    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042816    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042819    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042821    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:17:15.046427 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042824    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042827    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042830    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042833    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042836    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042839    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042841    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042844    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042846    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042850    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042853    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042856    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042858    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042861    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042864    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042866    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042869    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042871    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042875    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042879    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:17:15.046947 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042882    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042885    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042887    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042890    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042894    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042898    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042901    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042905    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042908    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042911    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042914    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042916    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042919    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042922    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042925    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042927    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042930    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042933    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042936    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:17:15.047438 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042938    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042941    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042946    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042949    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042952    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042955    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042957    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042960    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042964    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042967    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042969    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042972    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042975    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042977    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042980    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042982    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042985    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042988    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042991    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042993    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:17:15.047945 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042996    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.042999    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043001    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043004    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043006    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043009    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043011    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043073    2551 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043105    2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043111    2551 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043115    2551 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043121    2551 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043124    2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043128    2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043135    2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043138    2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043141    2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043145    2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043148    2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043151    2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043155    2551 flags.go:64] FLAG: --cgroup-root=""
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043158    2551 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043161    2551 flags.go:64] FLAG: --client-ca-file=""
Sep 10 13:17:15.048514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043164    2551 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043167    2551 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043172    2551 flags.go:64] FLAG: --cluster-domain=""
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043175    2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043178    2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043181    2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043185    2551 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043188    2551 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043191    2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043194    2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043197    2551 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043201    2551 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043204    2551 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043207    2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043210    2551 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043213    2551 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043217    2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043220    2551 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043223    2551 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043226    2551 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043229    2551 flags.go:64] FLAG: --enable-server="true"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043232    2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043236    2551 flags.go:64] FLAG: --event-burst="100"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043241    2551 flags.go:64] FLAG: --event-qps="50"
Sep 10 13:17:15.049087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043244    2551 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043247    2551 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043250    2551 flags.go:64] FLAG: --eviction-hard=""
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043253    2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043256    2551 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043260    2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043264    2551 flags.go:64] FLAG: --eviction-soft=""
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043267    2551 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043270    2551 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043273    2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043276    2551 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043279    2551 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043282    2551 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043285    2551 flags.go:64] FLAG: --feature-gates=""
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043289    2551 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043292    2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043295    2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043298    2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043301    2551 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043304    2551 flags.go:64] FLAG: --help="false"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043307    2551 flags.go:64] FLAG: --hostname-override="ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043310    2551 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043313    2551 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 13:17:15.049725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043316    2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043319    2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043323    2551 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043326    2551 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043328    2551 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043331    2551 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043334    2551 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043337    2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043342    2551 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043345    2551 flags.go:64] FLAG: --kube-reserved=""
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043348    2551 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043351    2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043354    2551 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043357    2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043361    2551 flags.go:64] FLAG: --lock-file=""
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043364    2551 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043367    2551 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043369    2551 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043375    2551 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043377    2551 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043380    2551 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043383    2551 flags.go:64] FLAG: --logging-format="text"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043386    2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043389    2551 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 13:17:15.050313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043392    2551 flags.go:64] FLAG: --manifest-url=""
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043395    2551 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043402    2551 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043405    2551 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043409    2551 flags.go:64] FLAG: --max-pods="110"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043412    2551 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043415    2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043418    2551 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043421    2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043424    2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043427    2551 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043430    2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043437    2551 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043440    2551 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043443    2551 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043446    2551 flags.go:64] FLAG: --pod-cidr=""
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043450    2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043455    2551 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043458    2551 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043461    2551 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043464    2551 flags.go:64] FLAG: --port="10250"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043467    2551 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043471    2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01cc581150ea41775"
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043474    2551 flags.go:64] FLAG: --qos-reserved=""
Sep 10 13:17:15.050927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043477    2551 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043480    2551 flags.go:64] FLAG: --register-node="true"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043483    2551 flags.go:64] FLAG: --register-with-taints=""
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043486    2551 flags.go:64] FLAG: --registry-burst="10"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043489    2551 flags.go:64] FLAG: --registry-qps="5"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043492    2551 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043494    2551 flags.go:64] FLAG: --reserved-memory=""
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043498    2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043501    2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043504    2551 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043507    2551 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043510    2551 flags.go:64] FLAG: --runonce="false"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043512    2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043516    2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043518    2551 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043521    2551 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043524    2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043527    2551 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043530    2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043533    2551 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043536    2551 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043539    2551 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043541    2551 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043544    2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043547    2551 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 13:17:15.051551 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043551    2551 flags.go:64] FLAG: --system-cgroups=""
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043554    2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043560    2551 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043563    2551 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043566    2551 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043572    2551 flags.go:64] FLAG: --tls-min-version=""
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043575    2551 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043578    2551 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043580    2551 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043583    2551 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043586    2551 flags.go:64] FLAG: --v="2"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043590    2551 flags.go:64] FLAG: --version="false"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043594    2551 flags.go:64] FLAG: --vmodule=""
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043598    2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.043601    2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043719    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043723    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043726    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043729    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043732    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043735    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043738    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:17:15.052171 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043743    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043746    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043749    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043752    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043755    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043758    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043761    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043763    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043766    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043768    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043771    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043774    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043777    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043779    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043782    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043786    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043789    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043792    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043794    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043797    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:17:15.052715 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043799    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043802    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043805    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043807    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043810    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043813    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043815    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043818    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043820    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043823    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043826    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043828    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043831    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043834    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043837    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043839    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043842    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043844    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043847    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043849    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:17:15.053445 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043852    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043854    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043857    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043860    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043863    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043866    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043868    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043872    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043875    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043878    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043881    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043883    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043886    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043888    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043891    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043893    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043896    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043898    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043901    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043903    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:17:15.054131 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043906    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043909    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043911    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043914    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043916    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043919    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043921    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043924    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043926    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043930    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043934    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043937    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043941    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043945    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043949    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043952    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043955    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043966    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043969    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:17:15.054640 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043973    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043976    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043979    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043982    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043985    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043987    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043990    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043994    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.043996    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.044008    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.044012    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.044016    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.044019    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.044024    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.050579    2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 13:17:15.055151 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.050593    2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.051313    2551 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.051320    2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051374    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051379    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051382    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051386    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051389    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051392    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051394    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051397    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051400    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051403    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051405    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051408    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051411    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051413    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051416    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051419    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:17:15.055578 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051421    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051424    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051427    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051429    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051433    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051437    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051440    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051443    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051446    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051449    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051452    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051454    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051457    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051462    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051465    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051467    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051470    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051472    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051475    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051478    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:17:15.056081 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051480    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051483    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051485    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051488    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051490    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051493    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051495    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051498    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051501    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051504    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051506    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051509    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051511    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051514    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051517    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051519    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051522    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051525    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051527    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051530    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051532    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:17:15.056609 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051535    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051538    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051540    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051543    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051546    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051551    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051554    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051557    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051560    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051564    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051568    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051572    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051575    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051577    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051580    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051582    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051585    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051587    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051590    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:17:15.057159 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051592    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051595    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051597    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051600    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051603    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051605    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051608    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051610    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051613    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051615    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051618    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051620    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051623    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051625    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051628    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051631    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051633    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051637    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051639    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:17:15.057627 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051642    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051645    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051647    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051650    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.051654    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051773    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051777    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051781    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051784    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051787    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051789    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051792    2551 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051795    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051798    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051800    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 13:17:15.058106 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051803    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051806    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051808    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051811    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051813    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051816    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051818    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051821    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051823    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051826    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051829    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051831    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051834    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051837    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051839    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051842    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051845    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051847    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051851    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 13:17:15.058480 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051854    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051856    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051858    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051861    2551 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051864    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051866    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051869    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051871    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051874    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051877    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051879    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051881    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051884    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051887    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051889    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051892    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051894    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051897    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051900    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 13:17:15.058980 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051904    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051907    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051909    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051912    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051914    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051917    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051919    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051922    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051924    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051927    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051931    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051934    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051937    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051941    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051943    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051946    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051949    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051952    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051954    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 13:17:15.059452 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051956    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051959    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051961    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051964    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051967    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051969    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051972    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051974    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051977    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051979    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051982    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051984    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051987    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051989    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051992    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051994    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.051998    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052001    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052004    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052007    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 13:17:15.059927 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052010    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052012    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052015    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052020    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052023    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052025    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052028    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052031    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052033    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052036    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052039    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:15.052041    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.052046    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.052138    2551 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.055481    2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 13:17:15.060419 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.056293    2551 server.go:1019] "Starting client certificate rotation"
Sep 10 13:17:15.060903 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.056390    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:17:15.060903 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.056427    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 13:17:15.060903 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.058601    2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 13:17:15.061026 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.061013    2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 13:17:15.071242 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.071226    2551 log.go:25] "Validated CRI v1 runtime API"
Sep 10 13:17:15.074083 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.074069    2551 log.go:25] "Validated CRI v1 image API"
Sep 10 13:17:15.074893 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.074879    2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 13:17:15.078216 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.078197    2551 fs.go:135] Filesystem UUIDs: map[14fb6ac0-9811-4ac3-bb48-b41da6fce744:/dev/nvme0n1p4 7672e2ea-8742-44a9-b01b-fd1a0a2b17e2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 10 13:17:15.078280 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.078216    2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 13:17:15.083707 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.083582    2551 manager.go:216] Machine: {Timestamp:2026-09-10 13:17:15.082233471 +0000 UTC m=+0.293426719 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113645 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b0aaa206805d1db0e103b97e3aff2 SystemUUID:ec2b0aaa-2068-05d1-db0e-103b97e3aff2 BootID:02ef6d5b-34e6-4978-8773-9e2b51e3a9cc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b0:64:3e:98:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b0:64:3e:98:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:3b:b8:32:60:1d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 13:17:15.083707 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.083702    2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 13:17:15.083810 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.083773    2551 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 13:17:15.085779 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.085764    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 13:17:15.086262 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.086241    2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 13:17:15.086389 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.086265    2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-9.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 13:17:15.086942 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.086933    2551 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 13:17:15.086982 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.086944    2551 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 13:17:15.086982 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.086955    2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 13:17:15.087562 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.087553    2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 13:17:15.088894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.088885    2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 13:17:15.092754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.092744    2551 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 13:17:15.092860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.092851    2551 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 13:17:15.092896 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.092878    2551 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 13:17:15.092896 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.092887    2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 13:17:15.092968 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.092897    2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 13:17:15.092968 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.092912    2551 kubelet.go:398] "Adding apiserver pod source"
Sep 10 13:17:15.092968 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.092944    2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 13:17:15.093377 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.093351    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 13:17:15.093485 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.093384    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 13:17:15.096720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.096699    2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 13:17:15.098993 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.098979    2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 13:17:15.099065 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099008    2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 13:17:15.099175 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099164    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 13:17:15.099221 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099180    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 13:17:15.099221 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099186    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 13:17:15.099221 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099191    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 13:17:15.099221 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099198    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 13:17:15.099221 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099203    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 13:17:15.099672 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099653    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 13:17:15.099704 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099676    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 13:17:15.099704 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099685    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 13:17:15.099704 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099691    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 13:17:15.099704 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099699    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 13:17:15.099813 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.099707    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 13:17:15.100431 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.100418    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 13:17:15.100431 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.100428    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 13:17:15.101935 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.101915    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-9.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 13:17:15.102601 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.102571    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-9.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 13:17:15.102654 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.102577    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 13:17:15.104231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.104218    2551 server.go:1268] "Started kubelet"
Sep 10 13:17:15.104337 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.104294    2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 13:17:15.104377 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.104357    2551 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 13:17:15.104377 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.104300    2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 13:17:15.104934 ip-10-0-137-9 systemd[1]: Started Kubernetes Kubelet.
Sep 10 13:17:15.106146 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.106131    2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 13:17:15.106330 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.106313    2551 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 13:17:15.109443 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.109425    2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 13:17:15.110724 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.109692    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-9.ec2.internal.18d3f81272229d81  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-9.ec2.internal,UID:ip-10-0-137-9.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-9.ec2.internal,},FirstTimestamp:2026-09-10 13:17:15.104189825 +0000 UTC m=+0.315383072,LastTimestamp:2026-09-10 13:17:15.104189825 +0000 UTC m=+0.315383072,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-9.ec2.internal,}"
Sep 10 13:17:15.114192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.114172    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 13:17:15.114835 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.114817    2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 13:17:15.115030 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.115012    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.115535 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115517    2551 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 13:17:15.115535 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115538    2551 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 13:17:15.115675 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115517    2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 13:17:15.115675 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115601    2551 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 13:17:15.115675 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115610    2551 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 13:17:15.115799 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115774    2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 13:17:15.115799 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115797    2551 factory.go:55] Registering systemd factory
Sep 10 13:17:15.115863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115802    2551 factory.go:223] Registration of the systemd container factory successfully
Sep 10 13:17:15.115979 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115968    2551 factory.go:153] Registering CRI-O factory
Sep 10 13:17:15.116039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115982    2551 factory.go:223] Registration of the crio container factory successfully
Sep 10 13:17:15.116039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.115995    2551 factory.go:103] Registering Raw factory
Sep 10 13:17:15.116039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.116004    2551 manager.go:1196] Started watching for new ooms in manager
Sep 10 13:17:15.116418 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.116403    2551 manager.go:349] Starting recovery of all containers
Sep 10 13:17:15.117529 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.117506    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 13:17:15.117705 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.117658    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-9.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 13:17:15.125756 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.125731    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5qn2s"
Sep 10 13:17:15.128134 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.128116    2551 manager.go:354] Recovery completed
Sep 10 13:17:15.133744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.133728    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5qn2s"
Sep 10 13:17:15.133814 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.133748    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:17:15.136028 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.136011    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:17:15.136093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.136040    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:17:15.136093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.136051    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:17:15.136631 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.136611    2551 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 13:17:15.136631 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.136629    2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 13:17:15.136791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.136645    2551 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 13:17:15.136791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.136648    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:17:15.137318 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.137305    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:17:15.137364 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.137329    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:17:15.137364 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.137341    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:17:15.138625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.138611    2551 policy_none.go:49] "None policy: Start"
Sep 10 13:17:15.138686 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.138630    2551 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 13:17:15.138686 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.138640    2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 13:17:15.140724 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.140708    2551 policy_none.go:47] "Start"
Sep 10 13:17:15.162344 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.162228    2551 manager.go:337] "Starting Device Plugin manager"
Sep 10 13:17:15.162457 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.162387    2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 13:17:15.162457 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.162400    2551 server.go:87] "Starting device plugin registration server"
Sep 10 13:17:15.162720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.162705    2551 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 13:17:15.162779 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.162722    2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 13:17:15.162856 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.162844    2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 13:17:15.162912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.162901    2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 13:17:15.162912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.162908    2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 13:17:15.163386 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.163363    2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 13:17:15.163470 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.163414    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.240849 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.240822    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 13:17:15.242915 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.241966    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 13:17:15.242915 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.242007    2551 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 13:17:15.242915 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.242029    2551 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 13:17:15.242915 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.242101    2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 13:17:15.243946 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.243928    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:17:15.262849 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.262832    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:17:15.263784 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.263768    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:17:15.263876 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.263800    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:17:15.263876 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.263818    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:17:15.263876 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.263847    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.269543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.269530    2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.269624 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.269550    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-9.ec2.internal\": node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.284800 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.284780    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.342971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.342907    2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal"]
Sep 10 13:17:15.343035 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.342977    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:17:15.343757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.343739    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:17:15.343820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.343765    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:17:15.343820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.343774    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:17:15.346041 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346030    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:17:15.346157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346144    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.346193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346175    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:17:15.346794 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346761    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:17:15.346794 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346802    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:17:15.346918 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346813    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:17:15.346953 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346813    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:17:15.346953 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346949    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:17:15.347013 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.346961    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:17:15.349056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.349041    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.349136 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.349070    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 13:17:15.349872 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.349845    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 13:17:15.349872 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.349873    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 13:17:15.349988 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.349886    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeHasSufficientPID"
Sep 10 13:17:15.374275 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.374253    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-9.ec2.internal\" not found" node="ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.378408 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.378393    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-9.ec2.internal\" not found" node="ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.385768 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.385752    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.417354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.417338    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/169e0ba168d588086edfb9d96364eff8-config\") pod \"kube-apiserver-proxy-ip-10-0-137-9.ec2.internal\" (UID: \"169e0ba168d588086edfb9d96364eff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.417450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.417360    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4edd218014deb42312be3a61dfdb05d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal\" (UID: \"4edd218014deb42312be3a61dfdb05d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.417450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.417389    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4edd218014deb42312be3a61dfdb05d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal\" (UID: \"4edd218014deb42312be3a61dfdb05d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.486223 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.486196    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.517565 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.517544    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4edd218014deb42312be3a61dfdb05d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal\" (UID: \"4edd218014deb42312be3a61dfdb05d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.517629 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.517563    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4edd218014deb42312be3a61dfdb05d2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal\" (UID: \"4edd218014deb42312be3a61dfdb05d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.517629 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.517582    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4edd218014deb42312be3a61dfdb05d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal\" (UID: \"4edd218014deb42312be3a61dfdb05d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.517629 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.517609    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/169e0ba168d588086edfb9d96364eff8-config\") pod \"kube-apiserver-proxy-ip-10-0-137-9.ec2.internal\" (UID: \"169e0ba168d588086edfb9d96364eff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.517751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.517631    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/169e0ba168d588086edfb9d96364eff8-config\") pod \"kube-apiserver-proxy-ip-10-0-137-9.ec2.internal\" (UID: \"169e0ba168d588086edfb9d96364eff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.517751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.517677    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4edd218014deb42312be3a61dfdb05d2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal\" (UID: \"4edd218014deb42312be3a61dfdb05d2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.586750 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.586731    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.677363 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.677315    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.681359 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:15.681336    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:15.687098 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.687067    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.787740 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.787716    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.888178 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.888147    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:15.993191 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:15.993141    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:16.058495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.058474    2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 13:17:16.058947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.058629    2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 13:17:16.095469 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:16.095439    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:16.112265 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.112247    2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:17:16.137744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.137700    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 13:12:15 +0000 UTC" deadline="2028-03-11 04:56:53.894949658 +0000 UTC"
Sep 10 13:17:16.137744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.137740    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13143h39m37.757213104s"
Sep 10 13:17:16.197888 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:16.197858    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:16.298329 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:16.298277    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-9.ec2.internal\" not found"
Sep 10 13:17:16.300764 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.300745    2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:17:16.315200 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.315179    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:16.327591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.327574    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 13:17:16.328601 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.328588    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal"
Sep 10 13:17:16.336227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.336213    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 13:17:16.343396 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.343379    2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:17:16.637989 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:16.637954    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4edd218014deb42312be3a61dfdb05d2.slice/crio-211c3eb8580d21a71819180faf414579ea580b88dc336639018c60c50de148e1 WatchSource:0}: Error finding container 211c3eb8580d21a71819180faf414579ea580b88dc336639018c60c50de148e1: Status 404 returned error can't find the container with id 211c3eb8580d21a71819180faf414579ea580b88dc336639018c60c50de148e1
Sep 10 13:17:16.642151 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:16.642136    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:17:16.759635 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:16.759606    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169e0ba168d588086edfb9d96364eff8.slice/crio-2173af7283556e334914ca2b8b7d15aa350ec9b6795d14398daab81d85f175b3 WatchSource:0}: Error finding container 2173af7283556e334914ca2b8b7d15aa350ec9b6795d14398daab81d85f175b3: Status 404 returned error can't find the container with id 2173af7283556e334914ca2b8b7d15aa350ec9b6795d14398daab81d85f175b3
Sep 10 13:17:17.096248 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.096229    2551 apiserver.go:52] "Watching apiserver"
Sep 10 13:17:17.105945 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.105916    2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 13:17:17.106327 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.106304    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-8xc7t","kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gr7fj","openshift-image-registry/node-ca-s8c85","openshift-multus/multus-additional-cni-plugins-xpm7s","openshift-multus/multus-mj7bz","openshift-multus/network-metrics-daemon-v5pcj","kube-system/konnectivity-agent-dkrjp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal","openshift-network-diagnostics/network-check-target-hzn5v","openshift-network-operator/iptables-alerter-l474j","openshift-ovn-kubernetes/ovnkube-node-z6jq7"]
Sep 10 13:17:17.109191 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.109163    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:17.109273 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.109254    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:17.111256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.111242    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.113499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.113482    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.114029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.114005    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 13:17:17.114170 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.114148    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 13:17:17.114220 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.114154    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 13:17:17.114220 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.114207    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dvrz6\""
Sep 10 13:17:17.114793 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.114769    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 13:17:17.115818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.115799    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.115898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.115857    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:17:17.116314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.116278    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 13:17:17.116433 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.116416    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nw6bg\""
Sep 10 13:17:17.118520 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.118500    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 13:17:17.118613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.118502    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 13:17:17.118613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.118506    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-86gg7\""
Sep 10 13:17:17.118768 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.118743    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.119012 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.118993    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 13:17:17.119113 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.119021    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 13:17:17.119311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.119257    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 13:17:17.120991 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.120975    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.121066 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.121025    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:17.121634 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.121601    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 13:17:17.121724 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.121633    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-857rb\""
Sep 10 13:17:17.123292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.123273    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.125717 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.125700    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.126126 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.126106    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nkkrx\""
Sep 10 13:17:17.126225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.126145    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 13:17:17.126290 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.126245    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 13:17:17.127240 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:17.127322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127245    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/37d43d0f-edb7-4379-994a-c64fb9c09f31-serviceca\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.127322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127262    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysctl-d\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127285    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysctl-conf\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127315    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-host\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127341    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.127549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127375    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69hq6\" (UniqueName: \"kubernetes.io/projected/ce2bd70f-e263-4dc3-9037-22dd49f9a637-kube-api-access-69hq6\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127427    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-system-cni-dir\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.127549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127485    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-modprobe-d\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127506    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.127549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127524    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.127549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127547    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-systemd\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127571    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-run\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127600    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-lib-modules\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127620    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbhm\" (UniqueName: \"kubernetes.io/projected/d3d5b9d7-255f-4b89-98b8-51328fd7d081-kube-api-access-jjbhm\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127649    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysconfig\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127699    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce2bd70f-e263-4dc3-9037-22dd49f9a637-tmp\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127723    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cnibin\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127751    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127778    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk7jh\" (UniqueName: \"kubernetes.io/projected/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-kube-api-access-qk7jh\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127813    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d43d0f-edb7-4379-994a-c64fb9c09f31-host\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127871    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf84m\" (UniqueName: \"kubernetes.io/projected/37d43d0f-edb7-4379-994a-c64fb9c09f31-kube-api-access-vf84m\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.127931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127925    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-kubernetes\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.128494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127943    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-tuned\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.128494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127958    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-sys\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.128494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.127991    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-var-lib-kubelet\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.128494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.128148    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jcw6z\""
Sep 10 13:17:17.128494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.128157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:17.128494 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.128223    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:17.129178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.129161    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 13:17:17.129307 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.129292    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 13:17:17.129639 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.129405    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 13:17:17.130771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.130731    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.131769 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.131738    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 13:17:17.133229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.133211    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.133386 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.133366    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d7tsw\""
Sep 10 13:17:17.133470 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.133412    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 13:17:17.133470 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.133419    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 13:17:17.133862 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.133847    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:17:17.135651 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.135636    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 13:17:17.135872 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.135860    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4gngz\""
Sep 10 13:17:17.136201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.136181    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 13:17:17.136292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.136187    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 13:17:17.136584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.136563    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 13:17:17.137048 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.137029    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 13:17:17.137144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.137104    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 13:17:17.151510 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.151492    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f6bcc"
Sep 10 13:17:17.160856 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.160836    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f6bcc"
Sep 10 13:17:17.217029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.217004    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 13:17:17.217238 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.217222    2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 13:17:17.229035 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229003    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-kubernetes\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229035 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229027    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-tuned\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229054    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229071    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229086    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-sys\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229101    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-systemd\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229135    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovnkube-script-lib\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229154    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229169    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/37d43d0f-edb7-4379-994a-c64fb9c09f31-serviceca\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229169    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-sys\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229191    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysctl-d\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229216    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysctl-conf\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229240    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-kubernetes\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.229294    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229370    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysctl-d\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229243    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v862h\" (UniqueName: \"kubernetes.io/projected/35be383a-b073-4bbd-acd3-52f08947f9d5-kube-api-access-v862h\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229401    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysctl-conf\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229426    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-kubelet\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.229441    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs podName:f19d0aa1-31e9-4f08-a729-6cb6332f34c7 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:17.729417185 +0000 UTC m=+2.940610446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs") pod "network-metrics-daemon-v5pcj" (UID: "f19d0aa1-31e9-4f08-a729-6cb6332f34c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229478    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-run-netns\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229507    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xdfx\" (UniqueName: \"kubernetes.io/projected/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-kube-api-access-7xdfx\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229538    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229546    2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229571    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69hq6\" (UniqueName: \"kubernetes.io/projected/ce2bd70f-e263-4dc3-9037-22dd49f9a637-kube-api-access-69hq6\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229598    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-k8s-cni-cncf-io\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229623    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-conf-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.229659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229663    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovn-node-metrics-cert\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229703    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-system-cni-dir\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229741    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-socket-dir-parent\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229759    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-system-cni-dir\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229765    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-netns\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229789    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35be383a-b073-4bbd-acd3-52f08947f9d5-host-slash\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229810    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-node-log\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229832    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229860    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-env-overrides\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229888    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-run\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229927    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-lib-modules\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.229978    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-run\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230042    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-etc-kubernetes\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230078    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-lib-modules\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230082    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lp5n\" (UniqueName: \"kubernetes.io/projected/214de4d4-a86b-4010-bc72-71eec5ab0676-kube-api-access-7lp5n\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230123    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fe469d7-9e50-4779-b9f0-e92eb262acb6-dbus\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230172    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-device-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.230292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230210    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-log-socket\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230250    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230293    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/37d43d0f-edb7-4379-994a-c64fb9c09f31-serviceca\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230303    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230326    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-multus-certs\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230342    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230367    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35be383a-b073-4bbd-acd3-52f08947f9d5-iptables-alerter-script\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230387    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-sys-fs\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230406    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-ovn\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230433    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-var-lib-kubelet\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230456    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230471    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-var-lib-kubelet\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230468    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230471    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98a61716-28c9-49e3-9121-9bbac2e21f08-agent-certs\") pod \"konnectivity-agent-dkrjp\" (UID: \"98a61716-28c9-49e3-9121-9bbac2e21f08\") " pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230518    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-socket-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230551    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-cni-netd\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230581    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-host\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.231198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230631    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-host\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230645    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230675    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-cnibin\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230728    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/214de4d4-a86b-4010-bc72-71eec5ab0676-cni-binary-copy\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230769    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-kubelet\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230794    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-etc-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230814    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98a61716-28c9-49e3-9121-9bbac2e21f08-konnectivity-ca\") pod \"konnectivity-agent-dkrjp\" (UID: \"98a61716-28c9-49e3-9121-9bbac2e21f08\") " pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230829    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-cni-bin\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230845    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-cni-multus\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230874    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-hostroot\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230901    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fe469d7-9e50-4779-b9f0-e92eb262acb6-kubelet-config\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230921    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-registration-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230936    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n2hk\" (UniqueName: \"kubernetes.io/projected/059a0081-933f-4ead-a876-f9f51ed86a78-kube-api-access-9n2hk\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230957    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-systemd-units\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230982    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-modprobe-d\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.230998    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.231980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231014    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231035    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-system-cni-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231100    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-modprobe-d\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231110    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-slash\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231144    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-systemd\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231162    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbhm\" (UniqueName: \"kubernetes.io/projected/d3d5b9d7-255f-4b89-98b8-51328fd7d081-kube-api-access-jjbhm\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231178    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-daemon-config\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231201    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-var-lib-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231221    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysconfig\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231243    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-systemd\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231252    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce2bd70f-e263-4dc3-9037-22dd49f9a637-tmp\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231296    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-sysconfig\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231446    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cnibin\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231532    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231556    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-cni-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231580    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cnibin\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231587    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-cni-bin\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.232459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231533    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-cni-binary-copy\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.233011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231612    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovnkube-config\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.233011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231641    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk7jh\" (UniqueName: \"kubernetes.io/projected/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-kube-api-access-qk7jh\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:17.233011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231668    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d43d0f-edb7-4379-994a-c64fb9c09f31-host\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.233011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231728    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf84m\" (UniqueName: \"kubernetes.io/projected/37d43d0f-edb7-4379-994a-c64fb9c09f31-kube-api-access-vf84m\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.233011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231747    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d43d0f-edb7-4379-994a-c64fb9c09f31-host\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.233011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.231887    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d3d5b9d7-255f-4b89-98b8-51328fd7d081-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.233192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.233054    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ce2bd70f-e263-4dc3-9037-22dd49f9a637-etc-tuned\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.233242 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.233224    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce2bd70f-e263-4dc3-9037-22dd49f9a637-tmp\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.239266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.239246    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69hq6\" (UniqueName: \"kubernetes.io/projected/ce2bd70f-e263-4dc3-9037-22dd49f9a637-kube-api-access-69hq6\") pod \"tuned-gr7fj\" (UID: \"ce2bd70f-e263-4dc3-9037-22dd49f9a637\") " pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.240797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.240781    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf84m\" (UniqueName: \"kubernetes.io/projected/37d43d0f-edb7-4379-994a-c64fb9c09f31-kube-api-access-vf84m\") pod \"node-ca-s8c85\" (UID: \"37d43d0f-edb7-4379-994a-c64fb9c09f31\") " pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.241403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.241378    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbhm\" (UniqueName: \"kubernetes.io/projected/d3d5b9d7-255f-4b89-98b8-51328fd7d081-kube-api-access-jjbhm\") pod \"multus-additional-cni-plugins-xpm7s\" (UID: \"d3d5b9d7-255f-4b89-98b8-51328fd7d081\") " pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.241530 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.241516    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk7jh\" (UniqueName: \"kubernetes.io/projected/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-kube-api-access-qk7jh\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:17.246411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.246374    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal" event={"ID":"169e0ba168d588086edfb9d96364eff8","Type":"ContainerStarted","Data":"2173af7283556e334914ca2b8b7d15aa350ec9b6795d14398daab81d85f175b3"}
Sep 10 13:17:17.247507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.247486    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal" event={"ID":"4edd218014deb42312be3a61dfdb05d2","Type":"ContainerStarted","Data":"211c3eb8580d21a71819180faf414579ea580b88dc336639018c60c50de148e1"}
Sep 10 13:17:17.332969 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.332945    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35be383a-b073-4bbd-acd3-52f08947f9d5-iptables-alerter-script\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.332969 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.332972    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-sys-fs\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.333185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.332989    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-ovn\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.333185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333013    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:17.333185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333038    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98a61716-28c9-49e3-9121-9bbac2e21f08-agent-certs\") pod \"konnectivity-agent-dkrjp\" (UID: \"98a61716-28c9-49e3-9121-9bbac2e21f08\") " pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.333185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333061    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-socket-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.333185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333067    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-sys-fs\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.333185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333085    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-cni-netd\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333235    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-socket-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333287    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-cni-netd\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333322    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-cnibin\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333370    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/214de4d4-a86b-4010-bc72-71eec5ab0676-cni-binary-copy\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333399    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-kubelet\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333437    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-etc-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333436    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-ovn\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333454    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98a61716-28c9-49e3-9121-9bbac2e21f08-konnectivity-ca\") pod \"konnectivity-agent-dkrjp\" (UID: \"98a61716-28c9-49e3-9121-9bbac2e21f08\") " pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333511    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-cnibin\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.333532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333514    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-cni-bin\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333560    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-etc-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333607    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-cni-multus\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333616    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-kubelet\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333647    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-hostroot\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333673    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fe469d7-9e50-4779-b9f0-e92eb262acb6-kubelet-config\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333697    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-registration-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333701    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-cni-bin\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333739    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9n2hk\" (UniqueName: \"kubernetes.io/projected/059a0081-933f-4ead-a876-f9f51ed86a78-kube-api-access-9n2hk\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333792    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fe469d7-9e50-4779-b9f0-e92eb262acb6-kubelet-config\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333784    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-systemd-units\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333845    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-system-cni-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333844    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-var-lib-cni-multus\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333774    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-hostroot\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333862    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-registration-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333904    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-systemd-units\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.333972    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-slash\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334101    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-system-cni-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.334291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334145    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-slash\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334154    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-daemon-config\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334183    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-var-lib-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334214    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-cni-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334217    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/98a61716-28c9-49e3-9121-9bbac2e21f08-konnectivity-ca\") pod \"konnectivity-agent-dkrjp\" (UID: \"98a61716-28c9-49e3-9121-9bbac2e21f08\") " pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334239    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-cni-bin\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334249    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/214de4d4-a86b-4010-bc72-71eec5ab0676-cni-binary-copy\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334299    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-var-lib-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334360    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-cni-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334369    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/35be383a-b073-4bbd-acd3-52f08947f9d5-iptables-alerter-script\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334371    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovnkube-config\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334403    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-cni-bin\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334438    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334467    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334497    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-systemd\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334524    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovnkube-script-lib\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334543    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.335323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334566    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v862h\" (UniqueName: \"kubernetes.io/projected/35be383a-b073-4bbd-acd3-52f08947f9d5-kube-api-access-v862h\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334600    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-kubelet\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334619    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-systemd\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334626    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-run-netns\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334638    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-run-openvswitch\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334686    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-run-netns\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334794    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-daemon-config\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334797    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-kubelet\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334887    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdfx\" (UniqueName: \"kubernetes.io/projected/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-kube-api-access-7xdfx\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334917    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334940    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-k8s-cni-cncf-io\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334960    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-conf-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.334984    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovn-node-metrics-cert\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335015    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-socket-dir-parent\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335046    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-netns\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335047    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovnkube-config\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335074    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35be383a-b073-4bbd-acd3-52f08947f9d5-host-slash\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335085    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-conf-dir\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.336353 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.335108    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335107    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-node-log\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335151    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-k8s-cni-cncf-io\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335175    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/35be383a-b073-4bbd-acd3-52f08947f9d5-host-slash\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335206    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-node-log\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335214    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-multus-socket-dir-parent\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.335266    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret podName:8fe469d7-9e50-4779-b9f0-e92eb262acb6 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:17.83522947 +0000 UTC m=+3.046422726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret") pod "global-pull-secret-syncer-8xc7t" (UID: "8fe469d7-9e50-4779-b9f0-e92eb262acb6") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335272    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-netns\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335287    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335365    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovnkube-script-lib\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335381    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-env-overrides\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335414    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335482    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-etc-kubernetes\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335523    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-etc-kubernetes\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335545    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lp5n\" (UniqueName: \"kubernetes.io/projected/214de4d4-a86b-4010-bc72-71eec5ab0676-kube-api-access-7lp5n\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335576    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fe469d7-9e50-4779-b9f0-e92eb262acb6-dbus\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335603    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-device-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.337256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335707    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-log-socket\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335739    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335795    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-multus-certs\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335822    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335833    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-log-socket\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335793    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fe469d7-9e50-4779-b9f0-e92eb262acb6-dbus\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335907    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-env-overrides\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335922    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/214de4d4-a86b-4010-bc72-71eec5ab0676-host-run-multus-certs\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335930    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-host-run-ovn-kubernetes\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335973    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.335979    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/059a0081-933f-4ead-a876-f9f51ed86a78-device-dir\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.337554    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/98a61716-28c9-49e3-9121-9bbac2e21f08-agent-certs\") pod \"konnectivity-agent-dkrjp\" (UID: \"98a61716-28c9-49e3-9121-9bbac2e21f08\") " pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.338078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.337816    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-ovn-node-metrics-cert\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.341530 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.341509    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:17:17.341633 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.341538    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:17:17.341633 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.341551    2551 projected.go:196] Error preparing data for projected volume kube-api-access-lrljz for pod openshift-network-diagnostics/network-check-target-hzn5v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:17.341803 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.341668    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz podName:89a62182-da82-4aa2-9356-476fbf0d9893 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:17.84165033 +0000 UTC m=+3.052843588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lrljz" (UniqueName: "kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz") pod "network-check-target-hzn5v" (UID: "89a62182-da82-4aa2-9356-476fbf0d9893") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:17.345519 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.345493    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v862h\" (UniqueName: \"kubernetes.io/projected/35be383a-b073-4bbd-acd3-52f08947f9d5-kube-api-access-v862h\") pod \"iptables-alerter-l474j\" (UID: \"35be383a-b073-4bbd-acd3-52f08947f9d5\") " pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.345710 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.345685    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdfx\" (UniqueName: \"kubernetes.io/projected/5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d-kube-api-access-7xdfx\") pod \"ovnkube-node-z6jq7\" (UID: \"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.345959 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.345919    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n2hk\" (UniqueName: \"kubernetes.io/projected/059a0081-933f-4ead-a876-f9f51ed86a78-kube-api-access-9n2hk\") pod \"aws-ebs-csi-driver-node-fxdm6\" (UID: \"059a0081-933f-4ead-a876-f9f51ed86a78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.346524 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.346465    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lp5n\" (UniqueName: \"kubernetes.io/projected/214de4d4-a86b-4010-bc72-71eec5ab0676-kube-api-access-7lp5n\") pod \"multus-mj7bz\" (UID: \"214de4d4-a86b-4010-bc72-71eec5ab0676\") " pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.423173 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.423152    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s8c85"
Sep 10 13:17:17.428538 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.428516    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gr7fj"
Sep 10 13:17:17.433777 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.433755    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d43d0f_edb7_4379_994a_c64fb9c09f31.slice/crio-612623350fdeff0c209a5fca70fa831d29ea1232c46f31776c60133fe66fd7b1 WatchSource:0}: Error finding container 612623350fdeff0c209a5fca70fa831d29ea1232c46f31776c60133fe66fd7b1: Status 404 returned error can't find the container with id 612623350fdeff0c209a5fca70fa831d29ea1232c46f31776c60133fe66fd7b1
Sep 10 13:17:17.437360 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.437343    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xpm7s"
Sep 10 13:17:17.437817 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.437797    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce2bd70f_e263_4dc3_9037_22dd49f9a637.slice/crio-0d87e030003939155a7535c55c0a5870bed3b70c816c886325b3250c5cb7b3db WatchSource:0}: Error finding container 0d87e030003939155a7535c55c0a5870bed3b70c816c886325b3250c5cb7b3db: Status 404 returned error can't find the container with id 0d87e030003939155a7535c55c0a5870bed3b70c816c886325b3250c5cb7b3db
Sep 10 13:17:17.444115 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.444093    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mj7bz"
Sep 10 13:17:17.447773 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.447734    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d5b9d7_255f_4b89_98b8_51328fd7d081.slice/crio-5b6a8a1c76a1dd8457fe9c01104caa26b3f8d364fcc2c0d0c2ff8086542d284d WatchSource:0}: Error finding container 5b6a8a1c76a1dd8457fe9c01104caa26b3f8d364fcc2c0d0c2ff8086542d284d: Status 404 returned error can't find the container with id 5b6a8a1c76a1dd8457fe9c01104caa26b3f8d364fcc2c0d0c2ff8086542d284d
Sep 10 13:17:17.448961 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.448805    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:17.453872 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.453852    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6"
Sep 10 13:17:17.454317 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.454297    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214de4d4_a86b_4010_bc72_71eec5ab0676.slice/crio-0b3a911a08c5d0beda9b857daf1af478025f9f3fe1cd896ea415ac00497d28ed WatchSource:0}: Error finding container 0b3a911a08c5d0beda9b857daf1af478025f9f3fe1cd896ea415ac00497d28ed: Status 404 returned error can't find the container with id 0b3a911a08c5d0beda9b857daf1af478025f9f3fe1cd896ea415ac00497d28ed
Sep 10 13:17:17.459180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.459159    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-l474j"
Sep 10 13:17:17.459286 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.459217    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a61716_28c9_49e3_9121_9bbac2e21f08.slice/crio-0db08dd4711acc2871ef0b3958b823ce282b5350ba5f61f4446adeeab13166b4 WatchSource:0}: Error finding container 0db08dd4711acc2871ef0b3958b823ce282b5350ba5f61f4446adeeab13166b4: Status 404 returned error can't find the container with id 0db08dd4711acc2871ef0b3958b823ce282b5350ba5f61f4446adeeab13166b4
Sep 10 13:17:17.463588 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.463563    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059a0081_933f_4ead_a876_f9f51ed86a78.slice/crio-a6b57b3332be3949a336ac883c898ffc6a6af93254e5a4acaaeea8645b6fa9cb WatchSource:0}: Error finding container a6b57b3332be3949a336ac883c898ffc6a6af93254e5a4acaaeea8645b6fa9cb: Status 404 returned error can't find the container with id a6b57b3332be3949a336ac883c898ffc6a6af93254e5a4acaaeea8645b6fa9cb
Sep 10 13:17:17.464291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.464261    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:17.469135 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.469113    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35be383a_b073_4bbd_acd3_52f08947f9d5.slice/crio-2bff3ae84d2da9900ec727f3addf9b1e5fbd63f5f35e1786c9263a2f795a1e4f WatchSource:0}: Error finding container 2bff3ae84d2da9900ec727f3addf9b1e5fbd63f5f35e1786c9263a2f795a1e4f: Status 404 returned error can't find the container with id 2bff3ae84d2da9900ec727f3addf9b1e5fbd63f5f35e1786c9263a2f795a1e4f
Sep 10 13:17:17.475017 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:17.474866    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7a8678_8fbf_44ad_ba97_dda31a9d9e5d.slice/crio-e08a155b60035f605cd7fdd1bc158dc0fa80ae70092970d3286d8eb14313242c WatchSource:0}: Error finding container e08a155b60035f605cd7fdd1bc158dc0fa80ae70092970d3286d8eb14313242c: Status 404 returned error can't find the container with id e08a155b60035f605cd7fdd1bc158dc0fa80ae70092970d3286d8eb14313242c
Sep 10 13:17:17.737944 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.737870    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:17.738086 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.737992    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:17.738086 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.738052    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs podName:f19d0aa1-31e9-4f08-a729-6cb6332f34c7 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:18.738037004 +0000 UTC m=+3.949230240 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs") pod "network-metrics-daemon-v5pcj" (UID: "f19d0aa1-31e9-4f08-a729-6cb6332f34c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:17.838787 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.838756    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:17.838928 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.838865    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:17.838928 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.838921    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret podName:8fe469d7-9e50-4779-b9f0-e92eb262acb6 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:18.83890332 +0000 UTC m=+4.050096578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret") pod "global-pull-secret-syncer-8xc7t" (UID: "8fe469d7-9e50-4779-b9f0-e92eb262acb6") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:17.939904 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:17.939732    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:17.940104 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.939909    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:17:17.940104 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.939937    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:17:17.940104 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.939948    2551 projected.go:196] Error preparing data for projected volume kube-api-access-lrljz for pod openshift-network-diagnostics/network-check-target-hzn5v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:17.940104 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:17.940002    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz podName:89a62182-da82-4aa2-9356-476fbf0d9893 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:18.939985564 +0000 UTC m=+4.151178800 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrljz" (UniqueName: "kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz") pod "network-check-target-hzn5v" (UID: "89a62182-da82-4aa2-9356-476fbf0d9893") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:18.164798 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.164736    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:12:17 +0000 UTC" deadline="2028-04-11 09:57:21.435391766 +0000 UTC"
Sep 10 13:17:18.164798 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.164774    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13892h40m3.27062286s"
Sep 10 13:17:18.243100 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.242363    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:18.243100 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.242514    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:18.243100 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.242957    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:18.243100 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.243056    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:18.251206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.251151    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l474j" event={"ID":"35be383a-b073-4bbd-acd3-52f08947f9d5","Type":"ContainerStarted","Data":"2bff3ae84d2da9900ec727f3addf9b1e5fbd63f5f35e1786c9263a2f795a1e4f"}
Sep 10 13:17:18.253222 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.253155    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6" event={"ID":"059a0081-933f-4ead-a876-f9f51ed86a78","Type":"ContainerStarted","Data":"a6b57b3332be3949a336ac883c898ffc6a6af93254e5a4acaaeea8645b6fa9cb"}
Sep 10 13:17:18.254825 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.254792    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s8c85" event={"ID":"37d43d0f-edb7-4379-994a-c64fb9c09f31","Type":"ContainerStarted","Data":"612623350fdeff0c209a5fca70fa831d29ea1232c46f31776c60133fe66fd7b1"}
Sep 10 13:17:18.256295 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.256212    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"e08a155b60035f605cd7fdd1bc158dc0fa80ae70092970d3286d8eb14313242c"}
Sep 10 13:17:18.257958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.257907    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dkrjp" event={"ID":"98a61716-28c9-49e3-9121-9bbac2e21f08","Type":"ContainerStarted","Data":"0db08dd4711acc2871ef0b3958b823ce282b5350ba5f61f4446adeeab13166b4"}
Sep 10 13:17:18.259491 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.259444    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mj7bz" event={"ID":"214de4d4-a86b-4010-bc72-71eec5ab0676","Type":"ContainerStarted","Data":"0b3a911a08c5d0beda9b857daf1af478025f9f3fe1cd896ea415ac00497d28ed"}
Sep 10 13:17:18.261015 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.260990    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerStarted","Data":"5b6a8a1c76a1dd8457fe9c01104caa26b3f8d364fcc2c0d0c2ff8086542d284d"}
Sep 10 13:17:18.262976 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.262952    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gr7fj" event={"ID":"ce2bd70f-e263-4dc3-9037-22dd49f9a637","Type":"ContainerStarted","Data":"0d87e030003939155a7535c55c0a5870bed3b70c816c886325b3250c5cb7b3db"}
Sep 10 13:17:18.746498 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.746188    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:18.748794 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.746659    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:18.748794 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.746758    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs podName:f19d0aa1-31e9-4f08-a729-6cb6332f34c7 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:20.746714072 +0000 UTC m=+5.957907330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs") pod "network-metrics-daemon-v5pcj" (UID: "f19d0aa1-31e9-4f08-a729-6cb6332f34c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:18.847303 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.847264    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:18.847478 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.847448    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:18.847541 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.847527    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret podName:8fe469d7-9e50-4779-b9f0-e92eb262acb6 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:20.847494714 +0000 UTC m=+6.058687969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret") pod "global-pull-secret-syncer-8xc7t" (UID: "8fe469d7-9e50-4779-b9f0-e92eb262acb6") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:18.948654 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:18.948451    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:18.948654 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.948631    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:17:18.948654 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.948655    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:17:18.948654 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.948668    2551 projected.go:196] Error preparing data for projected volume kube-api-access-lrljz for pod openshift-network-diagnostics/network-check-target-hzn5v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:18.949022 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:18.948753    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz podName:89a62182-da82-4aa2-9356-476fbf0d9893 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:20.948734136 +0000 UTC m=+6.159927390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrljz" (UniqueName: "kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz") pod "network-check-target-hzn5v" (UID: "89a62182-da82-4aa2-9356-476fbf0d9893") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:19.164983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:19.164914    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 13:12:17 +0000 UTC" deadline="2028-05-17 03:47:26.729967964 +0000 UTC"
Sep 10 13:17:19.164983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:19.164954    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14750h30m7.565019845s"
Sep 10 13:17:19.242844 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:19.242817    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:19.243021 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:19.242990    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:19.293697 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:19.293661    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal" event={"ID":"4edd218014deb42312be3a61dfdb05d2","Type":"ContainerStarted","Data":"01a1f22414eb87b876a2eaa0e9a361f604492d543e95169b9d879b73ac608c55"}
Sep 10 13:17:20.115310 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.114413    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-lp4ht"]
Sep 10 13:17:20.118389 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.117608    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.124501 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.124286    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 13:17:20.130867 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.130204    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 13:17:20.130867 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.130514    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9hbcs\""
Sep 10 13:17:20.157816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.157785    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvbks\" (UniqueName: \"kubernetes.io/projected/b578b265-6bda-4a01-b349-e0fbf48b6065-kube-api-access-fvbks\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.157945 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.157887    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b578b265-6bda-4a01-b349-e0fbf48b6065-hosts-file\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.158027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.157951    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b578b265-6bda-4a01-b349-e0fbf48b6065-tmp-dir\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.242585 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.242554    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:20.243162 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.242698    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:20.243162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.243127    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:20.243289 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.243235    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:20.258955 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.258925    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b578b265-6bda-4a01-b349-e0fbf48b6065-hosts-file\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.259078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.259001    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b578b265-6bda-4a01-b349-e0fbf48b6065-tmp-dir\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.259078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.259039    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvbks\" (UniqueName: \"kubernetes.io/projected/b578b265-6bda-4a01-b349-e0fbf48b6065-kube-api-access-fvbks\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.259418 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.259395    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b578b265-6bda-4a01-b349-e0fbf48b6065-hosts-file\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.259729 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.259707    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b578b265-6bda-4a01-b349-e0fbf48b6065-tmp-dir\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.275422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.275400    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvbks\" (UniqueName: \"kubernetes.io/projected/b578b265-6bda-4a01-b349-e0fbf48b6065-kube-api-access-fvbks\") pod \"node-resolver-lp4ht\" (UID: \"b578b265-6bda-4a01-b349-e0fbf48b6065\") " pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.303994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.303964    2551 generic.go:356] "Generic (PLEG): container finished" podID="4edd218014deb42312be3a61dfdb05d2" containerID="01a1f22414eb87b876a2eaa0e9a361f604492d543e95169b9d879b73ac608c55" exitCode=0
Sep 10 13:17:20.304123 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.304012    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal" event={"ID":"4edd218014deb42312be3a61dfdb05d2","Type":"ContainerDied","Data":"01a1f22414eb87b876a2eaa0e9a361f604492d543e95169b9d879b73ac608c55"}
Sep 10 13:17:20.434809 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.434773    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lp4ht"
Sep 10 13:17:20.763133 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.763098    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:20.763319 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.763276    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:20.763378 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.763339    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs podName:f19d0aa1-31e9-4f08-a729-6cb6332f34c7 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:24.763318367 +0000 UTC m=+9.974511623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs") pod "network-metrics-daemon-v5pcj" (UID: "f19d0aa1-31e9-4f08-a729-6cb6332f34c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:20.863688 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.863648    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:20.863894 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.863807    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:20.863894 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.863871    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret podName:8fe469d7-9e50-4779-b9f0-e92eb262acb6 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:24.863852661 +0000 UTC m=+10.075045914 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret") pod "global-pull-secret-syncer-8xc7t" (UID: "8fe469d7-9e50-4779-b9f0-e92eb262acb6") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:20.965111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:20.964478    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:20.965111 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.964633    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:17:20.965111 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.964654    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:17:20.965111 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.964669    2551 projected.go:196] Error preparing data for projected volume kube-api-access-lrljz for pod openshift-network-diagnostics/network-check-target-hzn5v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:20.965111 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:20.964756    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz podName:89a62182-da82-4aa2-9356-476fbf0d9893 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:24.964737514 +0000 UTC m=+10.175930771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrljz" (UniqueName: "kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz") pod "network-check-target-hzn5v" (UID: "89a62182-da82-4aa2-9356-476fbf0d9893") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:21.243947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:21.243415    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:21.243947 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:21.243565    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:22.242835 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:22.242800    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:22.243034 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:22.242949    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:22.243114 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:22.243071    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:22.243224 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:22.243200    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:23.243446 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:23.243415    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:23.243943 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:23.243574    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:24.242783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:24.242698    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:24.242949 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:24.242849    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:24.242949 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:24.242899    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:24.243078 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:24.243028    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:24.798428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:24.798372    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:24.799094 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:24.798604    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:24.799094 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:24.798685    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs podName:f19d0aa1-31e9-4f08-a729-6cb6332f34c7 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:32.798649331 +0000 UTC m=+18.009842588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs") pod "network-metrics-daemon-v5pcj" (UID: "f19d0aa1-31e9-4f08-a729-6cb6332f34c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:24.899093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:24.899061    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:24.899279 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:24.899193    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:24.899279 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:24.899258    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret podName:8fe469d7-9e50-4779-b9f0-e92eb262acb6 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:32.899240634 +0000 UTC m=+18.110433874 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret") pod "global-pull-secret-syncer-8xc7t" (UID: "8fe469d7-9e50-4779-b9f0-e92eb262acb6") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:25.000046 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:25.000017    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:25.000199 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:25.000178    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:17:25.000199 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:25.000199    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:17:25.000280 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:25.000211    2551 projected.go:196] Error preparing data for projected volume kube-api-access-lrljz for pod openshift-network-diagnostics/network-check-target-hzn5v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:25.000280 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:25.000264    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz podName:89a62182-da82-4aa2-9356-476fbf0d9893 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:33.000248892 +0000 UTC m=+18.211442167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrljz" (UniqueName: "kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz") pod "network-check-target-hzn5v" (UID: "89a62182-da82-4aa2-9356-476fbf0d9893") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:25.244432 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:25.244332    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:25.244586 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:25.244456    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:26.243276 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:26.243187    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:26.243713 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:26.243314    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:26.243713 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:26.243693    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:26.243828 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:26.243796    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:27.243267 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:27.243229    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:27.243426 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:27.243368    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:28.242978 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:28.242942    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:28.243102 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:28.243069    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:28.243164 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:28.243128    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:28.243223 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:28.243207    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:29.242440 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:29.242354    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:29.242922 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:29.242480    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:30.242883 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:30.242846    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:30.243308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:30.242863    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:30.243308 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:30.242971    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:30.243308 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:30.243056    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:31.242721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:31.242691    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:31.242886 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:31.242818    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:32.242223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:32.242183    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:32.242223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:32.242210    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:32.242867 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:32.242321    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:32.242867 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:32.242454    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:32.864582 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:32.864549    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:32.864775 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:32.864742    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:32.864842 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:32.864820    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs podName:f19d0aa1-31e9-4f08-a729-6cb6332f34c7 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:48.864799389 +0000 UTC m=+34.075992639 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs") pod "network-metrics-daemon-v5pcj" (UID: "f19d0aa1-31e9-4f08-a729-6cb6332f34c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:32.967851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:32.966042    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:32.967851 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:32.966239    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:32.967851 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:32.966314    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret podName:8fe469d7-9e50-4779-b9f0-e92eb262acb6 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:48.966294418 +0000 UTC m=+34.177487671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret") pod "global-pull-secret-syncer-8xc7t" (UID: "8fe469d7-9e50-4779-b9f0-e92eb262acb6") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:33.066929 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:33.066898    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:33.067102 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:33.067074    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:17:33.067102 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:33.067105    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:17:33.067268 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:33.067119    2551 projected.go:196] Error preparing data for projected volume kube-api-access-lrljz for pod openshift-network-diagnostics/network-check-target-hzn5v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:33.067268 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:33.067177    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz podName:89a62182-da82-4aa2-9356-476fbf0d9893 nodeName:}" failed. No retries permitted until 2026-09-10 13:17:49.067162514 +0000 UTC m=+34.278355762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrljz" (UniqueName: "kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz") pod "network-check-target-hzn5v" (UID: "89a62182-da82-4aa2-9356-476fbf0d9893") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:33.242853 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:33.242771    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:33.243216 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:33.242907    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:34.242572 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:34.242544    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:34.242758 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:34.242554    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:34.242758 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:34.242656    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:34.242863 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:34.242791    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:35.243268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:35.243242    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:35.243686 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:35.243325    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:36.242827 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:36.242792    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:36.242928 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:36.242906    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:36.242978 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:36.242941    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:36.243056 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:36.243033    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:36.245635 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:17:36.245614    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb578b265_6bda_4a01_b349_e0fbf48b6065.slice/crio-96b7dffe61b6c5fa57b714b29b96221740b1733c0f5c823e58db55063a643b04 WatchSource:0}: Error finding container 96b7dffe61b6c5fa57b714b29b96221740b1733c0f5c823e58db55063a643b04: Status 404 returned error can't find the container with id 96b7dffe61b6c5fa57b714b29b96221740b1733c0f5c823e58db55063a643b04
Sep 10 13:17:36.339895 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:36.339509    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lp4ht" event={"ID":"b578b265-6bda-4a01-b349-e0fbf48b6065","Type":"ContainerStarted","Data":"96b7dffe61b6c5fa57b714b29b96221740b1733c0f5c823e58db55063a643b04"}
Sep 10 13:17:37.242917 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.242730    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:37.243029 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:37.243006    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:37.343408 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.343327    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal" event={"ID":"4edd218014deb42312be3a61dfdb05d2","Type":"ContainerStarted","Data":"c849e4cc065cd52afb7df070409237bac5a8ec21fb69f9e723082b714d704c73"}
Sep 10 13:17:37.349208 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349189    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:17:37.349624 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349599    2551 generic.go:356] "Generic (PLEG): container finished" podID="5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d" containerID="60f0c4a8e251452b1214b381e58ee48cbdefceb57fdc14797b5525091d40a1ba" exitCode=1
Sep 10 13:17:37.349806 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349707    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"b14330ee301a433de0d1c21c6bf9f1fcd4cd5b2c9ff6c7c30a9228f7dcb6f059"}
Sep 10 13:17:37.349806 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349784    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"eb88996d49c5756d4efa0d0795de7628d7675388823aad0843691b19666fce05"}
Sep 10 13:17:37.349921 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349832    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"03abce7f1811087d20d786dcae9d72acd820d9ed6094d4db23847471d1cde9a2"}
Sep 10 13:17:37.349921 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349882    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"b99033cd0f1cf6b96d124e3ab45098b20a20abfc3d36fee2a9aa3b3c7b2d86d5"}
Sep 10 13:17:37.350029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349932    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerDied","Data":"60f0c4a8e251452b1214b381e58ee48cbdefceb57fdc14797b5525091d40a1ba"}
Sep 10 13:17:37.350029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.349986    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"f752ea92f0756f8c8e947c863d4c41896e14aaac83c2b1f4fee4c070b56832a8"}
Sep 10 13:17:37.351436 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.351406    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dkrjp" event={"ID":"98a61716-28c9-49e3-9121-9bbac2e21f08","Type":"ContainerStarted","Data":"3cd4693662746e154b2bec1190dfa8b3462736a03ac341b1d7d9ebe8a6096269"}
Sep 10 13:17:37.353031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.353001    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mj7bz" event={"ID":"214de4d4-a86b-4010-bc72-71eec5ab0676","Type":"ContainerStarted","Data":"aeca3a66898c48d253cb5785a2b19eba824a2483b4e98a04f2b47742c3d393d4"}
Sep 10 13:17:37.354737 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.354712    2551 generic.go:356] "Generic (PLEG): container finished" podID="d3d5b9d7-255f-4b89-98b8-51328fd7d081" containerID="8b79278c79f7deaba899ba7fae8cb9b7bab55e826aab48cd5daf61837ea4539e" exitCode=0
Sep 10 13:17:37.354819 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.354787    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerDied","Data":"8b79278c79f7deaba899ba7fae8cb9b7bab55e826aab48cd5daf61837ea4539e"}
Sep 10 13:17:37.357002 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.356981    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gr7fj" event={"ID":"ce2bd70f-e263-4dc3-9037-22dd49f9a637","Type":"ContainerStarted","Data":"71cc4bb0d61253f04bca89e1ccca997373fe348104250aa29a7d99b0e256cdb3"}
Sep 10 13:17:37.359017 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.358999    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lp4ht" event={"ID":"b578b265-6bda-4a01-b349-e0fbf48b6065","Type":"ContainerStarted","Data":"dc7e95fb79792c2d9da4d06770d58595ce2cd055380ade5c49f7133ad3c0c922"}
Sep 10 13:17:37.360587 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.360565    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal" event={"ID":"169e0ba168d588086edfb9d96364eff8","Type":"ContainerStarted","Data":"2a90ec1359ba63beb0b9e14f983a8573ed85a737d90b8250a463554693024456"}
Sep 10 13:17:37.364809 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.364783    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6" event={"ID":"059a0081-933f-4ead-a876-f9f51ed86a78","Type":"ContainerStarted","Data":"1a47605e72cd2b71210bd0c65f38c90a39d1c617abac9fbf6e5b3b38a2907b5e"}
Sep 10 13:17:37.366575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.366552    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s8c85" event={"ID":"37d43d0f-edb7-4379-994a-c64fb9c09f31","Type":"ContainerStarted","Data":"d5ab576f79434319ac6db951b86fa9184ecfc987bf8c465db622bb2477176686"}
Sep 10 13:17:37.381963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.381913    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-9.ec2.internal" podStartSLOduration=21.381896689 podStartE2EDuration="21.381896689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:37.364207322 +0000 UTC m=+22.575400613" watchObservedRunningTime="2026-09-10 13:17:37.381896689 +0000 UTC m=+22.593089948"
Sep 10 13:17:37.382261 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.382235    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-9.ec2.internal" podStartSLOduration=21.382226433 podStartE2EDuration="21.382226433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:37.381562799 +0000 UTC m=+22.592756059" watchObservedRunningTime="2026-09-10 13:17:37.382226433 +0000 UTC m=+22.593419690"
Sep 10 13:17:37.402482 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.402440    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gr7fj" podStartSLOduration=2.571886343 podStartE2EDuration="21.402428859s" totalImagesPullingTime="18.830542516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:37.402087516 +0000 UTC m=+22.613280776" watchObservedRunningTime="2026-09-10 13:17:37.402428859 +0000 UTC m=+22.613622143"
Sep 10 13:17:37.446957 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.446917    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mj7bz" podStartSLOduration=2.590287329 podStartE2EDuration="21.446905195s" totalImagesPullingTime="18.856617866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:37.42398613 +0000 UTC m=+22.635179388" watchObservedRunningTime="2026-09-10 13:17:37.446905195 +0000 UTC m=+22.658098455"
Sep 10 13:17:37.463563 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.463519    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dkrjp" podStartSLOduration=10.703944837 podStartE2EDuration="21.463504347s" totalImagesPullingTime="10.75955951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:37.462913648 +0000 UTC m=+22.674106906" watchObservedRunningTime="2026-09-10 13:17:37.463504347 +0000 UTC m=+22.674697604"
Sep 10 13:17:37.478950 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.478911    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lp4ht" podStartSLOduration=17.47889375 podStartE2EDuration="17.47889375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:37.478622116 +0000 UTC m=+22.689815376" watchObservedRunningTime="2026-09-10 13:17:37.47889375 +0000 UTC m=+22.690087008"
Sep 10 13:17:37.569515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.569495    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:37.647818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.647794    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:37.726579 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:37.726557    2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 13:17:38.175019 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.174915    2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T13:17:37.726574226Z","UUID":"149d85fa-81a0-43ed-bcc2-3344573ac699","Handler":null,"Name":"","Endpoint":""}
Sep 10 13:17:38.176320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.176293    2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 13:17:38.176453 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.176328    2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 13:17:38.242425 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.242400    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:38.242565 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.242400    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:38.242565 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:38.242505    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:38.242705 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:38.242584    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:38.370219 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.370149    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-l474j" event={"ID":"35be383a-b073-4bbd-acd3-52f08947f9d5","Type":"ContainerStarted","Data":"65752b6f36c8dcddbd1ed92aa094f5fa0da4992436b359be39319240b1ce79b9"}
Sep 10 13:17:38.372752 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.372720    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6" event={"ID":"059a0081-933f-4ead-a876-f9f51ed86a78","Type":"ContainerStarted","Data":"923309b29a26c68ea01ba671d94181354c0b5c000187eb13a5999b38409936d7"}
Sep 10 13:17:38.389595 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.389544    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s8c85" podStartSLOduration=4.843209237 podStartE2EDuration="23.389527425s" totalImagesPullingTime="18.546318188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:37.503047326 +0000 UTC m=+22.714240584" watchObservedRunningTime="2026-09-10 13:17:38.389527425 +0000 UTC m=+23.600720683"
Sep 10 13:17:38.547834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.547789    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:38.648989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:38.648956    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:39.027995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.027961    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:39.124500 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.124469    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:39.225539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.225475    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:39.242676 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.242637    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:39.242809 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:39.242784    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:39.321806 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.321771    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:39.377254 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.377232    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6" event={"ID":"059a0081-933f-4ead-a876-f9f51ed86a78","Type":"ContainerStarted","Data":"84e9b9475c16668af3d4e1ce58105fc14b09d99a17c97e5557e1d50d324b9751"}
Sep 10 13:17:39.397652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.397617    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-l474j" podStartSLOduration=4.88728967 podStartE2EDuration="23.397602851s" totalImagesPullingTime="18.510313181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:38.389372457 +0000 UTC m=+23.600565726" watchObservedRunningTime="2026-09-10 13:17:39.397602851 +0000 UTC m=+24.608796108"
Sep 10 13:17:39.526062 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.526032    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:39.777811 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.777736    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:39.877600 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:39.877569    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:40.242998 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:40.242792    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:40.243126 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:40.242791    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:40.243126 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:40.243019    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:40.243126 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:40.243075    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:40.381554 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:40.381527    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:17:40.381963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:40.381942    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"f5134d90c6a84f9f5755159d64d9e906b2c577d463ff2ce2be4bc2f2f367bb2f"}
Sep 10 13:17:41.111008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.110972    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:41.111838 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.111818    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:41.128712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.128653    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxdm6" podStartSLOduration=4.03532314 podStartE2EDuration="25.128632077s" totalImagesPullingTime="21.093308937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:39.397595067 +0000 UTC m=+24.608788323" watchObservedRunningTime="2026-09-10 13:17:41.128632077 +0000 UTC m=+26.339825330"
Sep 10 13:17:41.242982 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.242949    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:41.243125 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:41.243074    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:41.384873 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.384792    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:41.385298 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.385282    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dkrjp"
Sep 10 13:17:41.443587 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.443562    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:41.543927 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.543896    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:41.673862 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:41.673785    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lp4ht_b578b265-6bda-4a01-b349-e0fbf48b6065/dns-node-resolver/0.log"
Sep 10 13:17:42.242751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:42.242724    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:42.242910 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:42.242729    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:42.242910 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:42.242817    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:42.243006 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:42.242905    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:42.263056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:42.263026    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:42.272555 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:42.272534    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s8c85_37d43d0f-edb7-4379-994a-c64fb9c09f31/node-ca/0.log"
Sep 10 13:17:43.242973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.242785    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:43.243292 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:43.243016    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:43.391714 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.391692    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:17:43.392091 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.392066    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"e5ad60960dd33b6c9dc9494d9366c94da73dc60e4f47c34df1b8c003f0cd5487"}
Sep 10 13:17:43.392317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.392298    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:43.392507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.392490    2551 scope.go:117] "RemoveContainer" containerID="60f0c4a8e251452b1214b381e58ee48cbdefceb57fdc14797b5525091d40a1ba"
Sep 10 13:17:43.394051 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.394034    2551 generic.go:356] "Generic (PLEG): container finished" podID="d3d5b9d7-255f-4b89-98b8-51328fd7d081" containerID="2fa49937d7511756c8218b6cffd13001123170a815ea68d8f627865cbaaf9c74" exitCode=0
Sep 10 13:17:43.394155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.394109    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerDied","Data":"2fa49937d7511756c8218b6cffd13001123170a815ea68d8f627865cbaaf9c74"}
Sep 10 13:17:43.408079 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.408052    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:43.798509 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.798476    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:43.898695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.898651    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:43.999585 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:43.999556    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:44.119235 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.119160    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:44.243041 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.243012    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:44.243388 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:44.243106    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:44.243388 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.243156    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:44.243388 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:44.243234    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:44.370197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.370126    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:44.399166 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.399147    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:17:44.399467 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.399450    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" event={"ID":"5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d","Type":"ContainerStarted","Data":"fc414bbb7f19f99647cab3ab43c9d5f9fa9defe5fef31a9115d107ac0b70cfdc"}
Sep 10 13:17:44.399548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.399536    2551 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 13:17:44.399788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.399769    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:44.413815 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.413797    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:44.435377 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.435339    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" podStartSLOduration=9.123258163 podStartE2EDuration="28.435327148s" totalImagesPullingTime="19.312068985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:44.433777844 +0000 UTC m=+29.644971100" watchObservedRunningTime="2026-09-10 13:17:44.435327148 +0000 UTC m=+29.646520404"
Sep 10 13:17:44.633290 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:44.633270    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:45.114255 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:45.114220    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:45.244352 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:45.244325    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:45.244743 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:45.244432    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:45.403138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:45.403062    2551 generic.go:356] "Generic (PLEG): container finished" podID="d3d5b9d7-255f-4b89-98b8-51328fd7d081" containerID="b36a2ce9ab976cccdc3967f07d7f64573668b574377a4e79de64db9dbce191d4" exitCode=0
Sep 10 13:17:45.403286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:45.403145    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerDied","Data":"b36a2ce9ab976cccdc3967f07d7f64573668b574377a4e79de64db9dbce191d4"}
Sep 10 13:17:45.403286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:45.403233    2551 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 13:17:45.693291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:45.693228    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:46.153235 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:46.153179    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:46.242645 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:46.242624    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:46.242760 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:46.242719    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:46.242808 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:46.242775    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:46.242881 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:46.242864    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:46.405319 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:46.405258    2551 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 13:17:46.482925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:46.482894    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:46.684239 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:46.684029    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:46.891702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:46.891680    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:17:47.105887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.105860    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:47.242450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.242418    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:47.242562 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:47.242518    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:47.409912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.409849    2551 generic.go:356] "Generic (PLEG): container finished" podID="d3d5b9d7-255f-4b89-98b8-51328fd7d081" containerID="438dddecb2e9f67324f7d300175a3de54ee0297817b259166bf170e8b3b78792" exitCode=0
Sep 10 13:17:47.410312 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.409928    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerDied","Data":"438dddecb2e9f67324f7d300175a3de54ee0297817b259166bf170e8b3b78792"}
Sep 10 13:17:47.419782 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.419752    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7" podUID="5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 10 13:17:47.694016 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.693953    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:47.811322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.811294    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:47.917291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:47.917263    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:48.242558 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.242525    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:48.242750 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:48.242647    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:48.242750 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.242694    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:48.242843 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:48.242797    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:48.298200 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.298171    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:48.483711 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.483678    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:48.544961 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.544933    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:48.693646 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.693607    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:48.886239 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.886119    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:48.890166 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.890135    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:48.890290 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:48.890272    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:48.890338 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:48.890329    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs podName:f19d0aa1-31e9-4f08-a729-6cb6332f34c7 nodeName:}" failed. No retries permitted until 2026-09-10 13:18:20.890315804 +0000 UTC m=+66.101509044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs") pod "network-metrics-daemon-v5pcj" (UID: "f19d0aa1-31e9-4f08-a729-6cb6332f34c7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 13:17:48.990497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:48.990461    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:48.990660 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:48.990614    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:48.990749 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:48.990700    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret podName:8fe469d7-9e50-4779-b9f0-e92eb262acb6 nodeName:}" failed. No retries permitted until 2026-09-10 13:18:20.990681699 +0000 UTC m=+66.201874949 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret") pod "global-pull-secret-syncer-8xc7t" (UID: "8fe469d7-9e50-4779-b9f0-e92eb262acb6") : object "kube-system"/"original-pull-secret" not registered
Sep 10 13:17:49.090954 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:49.090917    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:49.091255 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:49.091082    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 13:17:49.091255 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:49.091108    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 13:17:49.091255 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:49.091119    2551 projected.go:196] Error preparing data for projected volume kube-api-access-lrljz for pod openshift-network-diagnostics/network-check-target-hzn5v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:49.091255 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:49.091172    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz podName:89a62182-da82-4aa2-9356-476fbf0d9893 nodeName:}" failed. No retries permitted until 2026-09-10 13:18:21.091153594 +0000 UTC m=+66.302346849 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrljz" (UniqueName: "kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz") pod "network-check-target-hzn5v" (UID: "89a62182-da82-4aa2-9356-476fbf0d9893") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 13:17:49.246338 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:49.246267    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:49.246488 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:49.246366    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:49.680845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:49.680811    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:50.081784 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:50.081753    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:50.242757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:50.242731    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:50.242964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:50.242734    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:50.243083 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:50.242957    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:50.243083 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:50.242833    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:50.290329 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:50.290306    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:50.390791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:50.390719    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:50.708837 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:50.708760    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:50.885412 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:50.885380    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:51.242808 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:51.242781    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:51.242973 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:51.242882    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:51.294018 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:51.293988    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:51.539723 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:51.539688    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:51.824055 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:51.823983    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:51.899453 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:51.899422    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:52.243165 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:52.243088    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:52.243318 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:52.243095    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:52.243318 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:52.243213    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:52.243318 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:52.243267    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:52.481139 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:52.481107    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:52.682851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:52.682826    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:53.242906 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:53.242875    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:53.243304 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:53.242988    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:53.307309 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:53.307280    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:53.425994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:53.425968    2551 generic.go:356] "Generic (PLEG): container finished" podID="d3d5b9d7-255f-4b89-98b8-51328fd7d081" containerID="c60f4b5ea972a7fa39cf5339953fca4084e6869c21956e5af4917d2496212efa" exitCode=0
Sep 10 13:17:53.426152 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:53.426012    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerDied","Data":"c60f4b5ea972a7fa39cf5339953fca4084e6869c21956e5af4917d2496212efa"}
Sep 10 13:17:53.521224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:53.521203    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:53.879391 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:53.879363    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:54.243132 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:54.243054    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:54.243494 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:54.243146    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:54.243494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:54.243056    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:54.243494 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:54.243214    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:54.282072 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:54.282047    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:54.433052 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:54.433024    2551 generic.go:356] "Generic (PLEG): container finished" podID="d3d5b9d7-255f-4b89-98b8-51328fd7d081" containerID="5b1a2a1fbc63613434cb6670633f2072698156c4d47003c53a8a72d149b53522" exitCode=0
Sep 10 13:17:54.433194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:54.433066    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerDied","Data":"5b1a2a1fbc63613434cb6670633f2072698156c4d47003c53a8a72d149b53522"}
Sep 10 13:17:54.486792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:54.486740    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:55.243659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:55.243631    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:55.244125 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:55.243719    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:55.440702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:55.440657    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" event={"ID":"d3d5b9d7-255f-4b89-98b8-51328fd7d081","Type":"ContainerStarted","Data":"ffbeb940a97d27b0faa2660ccde24310c37a2ee35d19977a5f9702b95a5315f0"}
Sep 10 13:17:55.486773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:55.486733    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xpm7s" podStartSLOduration=7.70501065 podStartE2EDuration="39.486719839s" totalImagesPullingTime="31.781709189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:17:55.485563974 +0000 UTC m=+40.696757230" watchObservedRunningTime="2026-09-10 13:17:55.486719839 +0000 UTC m=+40.697913095"
Sep 10 13:17:56.242641 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:56.242608    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:56.242641 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:56.242639    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:56.242845 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:56.242736    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:56.242889 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:56.242852    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:57.245581 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:57.245553    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:57.245929 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:57.245642    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:58.242192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:58.242162    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:17:58.242376 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:58.242281    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:17:58.242376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:58.242318    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:17:58.242478 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:58.242389    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:17:58.795208 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:58.795171    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:59.243052 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:59.242973    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:17:59.243209 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:17:59.243087    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:17:59.512040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:59.511962    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:59.613788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:59.613761    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:17:59.788188 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:17:59.788167    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:00.171644 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.171573    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:00.242519 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.242496    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:00.242624 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.242499    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:00.242624 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:00.242590    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:00.242733 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:00.242649    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:00.369796 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.369770    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:00.471546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.471497    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:00.571700 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.571680    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:00.672881 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.672856    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:00.741154 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.741105    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:00.990322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:00.990294    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:01.185493 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:01.185459    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:01.242553 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:01.242530    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:01.242699 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:01.242659    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:01.621905 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:01.621876    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:02.057913 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:02.057890    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:02.126569 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:02.126543    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:02.242870 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:02.242841    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:02.243210 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:02.242880    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:02.243210 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:02.242991    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:02.243210 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:02.243020    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:03.242340 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:03.242311    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:03.242512 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:03.242407    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:04.242565 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:04.242535    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:04.242932 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:04.242535    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:04.242932 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:04.242622    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:04.242932 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:04.242717    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:05.244327 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:05.244299    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:05.244741 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:05.244406    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:05.575106 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:05.575077    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:05.640367 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:05.640343    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:06.242583 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:06.242547    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:06.242761 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:06.242651    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:06.242761 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:06.242689    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:06.242843 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:06.242779    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:07.243002 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:07.242970    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:07.243367 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:07.243094    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:08.232242 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:08.232210    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:08.243208 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:08.243190    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:08.243475 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:08.243191    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:08.243475 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:08.243350    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:08.243475 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:08.243271    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:08.339556 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:08.339531    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:08.449379 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:08.449354    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:08.653959 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:08.653929    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:09.242866 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:09.242840    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:09.243085 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:09.242930    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:10.242196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:10.242168    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:10.242544 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:10.242168    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:10.242544 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:10.242265    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:10.242544 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:10.242340    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:11.104366 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:11.104333    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:11.197708 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:11.197683    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:11.242746 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:11.242728    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:11.243006 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:11.242817    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:11.299370 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:11.299346    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:11.400389 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:11.400337    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:11.556998 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:11.556976    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:12.242776 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.242598    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:12.243125 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.242620    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:12.243125 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:12.242862    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:12.243125 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:12.242925    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:12.441358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.441330    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5pcj"]
Sep 10 13:18:12.443122 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.443101    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8xc7t"]
Sep 10 13:18:12.443247 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.443220    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:12.443345 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:12.443318    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:12.443708 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.443691    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzn5v"]
Sep 10 13:18:12.475179 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.475102    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:12.475179 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:12.475113    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:12.475311 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:12.475192    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:12.475352 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:12.475315    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:14.242184 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.242154    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:14.242580 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.242237    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:14.242580 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:14.242242    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hzn5v" podUID="89a62182-da82-4aa2-9356-476fbf0d9893"
Sep 10 13:18:14.242580 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.242246    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:14.242580 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:14.242312    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8xc7t" podUID="8fe469d7-9e50-4779-b9f0-e92eb262acb6"
Sep 10 13:18:14.242580 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:14.242398    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5pcj" podUID="f19d0aa1-31e9-4f08-a729-6cb6332f34c7"
Sep 10 13:18:14.270728 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.270705    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:14.368162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.368114    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:14.469291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.469265    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:14.533512 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.533473    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:14.838354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:14.838253    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:15.276369 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:15.276223    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:15.376214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:15.376180    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:15.938887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:15.938867    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:16.003424 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.003389    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:16.059411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.059390    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-9.ec2.internal" event="NodeReady"
Sep 10 13:18:16.059516 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.059505    2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 13:18:16.098463 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.098443    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fc87d7f97-48xl2"]
Sep 10 13:18:16.100893 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.100879    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.103855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.103837    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 13:18:16.103942 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.103902    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 13:18:16.104037 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.104024    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hqw92\""
Sep 10 13:18:16.104101 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.104087    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 13:18:16.109401 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.109382    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 13:18:16.113484 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.113467    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mfbgb"]
Sep 10 13:18:16.115631 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.115612    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.115825 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.115808    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wdq7m"]
Sep 10 13:18:16.118395 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.118375    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 13:18:16.118600 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.118524    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 13:18:16.118600 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.118526    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 13:18:16.118900 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.118883    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.119659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.119636    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-74kfd\""
Sep 10 13:18:16.122465 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.121098    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fc87d7f97-48xl2"]
Sep 10 13:18:16.122465 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.122253    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 13:18:16.124336 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.122818    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 13:18:16.124336 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.123074    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fkjjg\""
Sep 10 13:18:16.128449 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.128428    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mfbgb"]
Sep 10 13:18:16.131424 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.131405    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wdq7m"]
Sep 10 13:18:16.192239 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192188    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9748123d-df56-462f-a50f-6bfd6d565527-config-volume\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.192239 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192222    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2w9c\" (UniqueName: \"kubernetes.io/projected/0d3bc038-4830-456d-b704-ddedf5e5d2d2-kube-api-access-x2w9c\") pod \"ingress-canary-mfbgb\" (UID: \"0d3bc038-4830-456d-b704-ddedf5e5d2d2\") " pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.192368 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192250    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-registry-tls\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.192368 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192280    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2bb7f1fa-465a-43b3-8884-962f45148f07-registry-certificates\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.192368 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192305    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bb7f1fa-465a-43b3-8884-962f45148f07-trusted-ca\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.192494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192396    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d3bc038-4830-456d-b704-ddedf5e5d2d2-cert\") pod \"ingress-canary-mfbgb\" (UID: \"0d3bc038-4830-456d-b704-ddedf5e5d2d2\") " pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.192494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192413    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9748123d-df56-462f-a50f-6bfd6d565527-metrics-tls\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.192494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192428    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2bb7f1fa-465a-43b3-8884-962f45148f07-ca-trust-extracted\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.192494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192447    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-bound-sa-token\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.192712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192503    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2bb7f1fa-465a-43b3-8884-962f45148f07-image-registry-private-configuration\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.192712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192538    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j266c\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-kube-api-access-j266c\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.192712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192581    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9748123d-df56-462f-a50f-6bfd6d565527-tmp-dir\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.192712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192604    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64pb4\" (UniqueName: \"kubernetes.io/projected/9748123d-df56-462f-a50f-6bfd6d565527-kube-api-access-64pb4\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.192712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.192628    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2bb7f1fa-465a-43b3-8884-962f45148f07-installation-pull-secrets\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.212200 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.212182    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gbsvc"]
Sep 10 13:18:16.214729 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.214714    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.217432 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.217410    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rwkt7\""
Sep 10 13:18:16.217523 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.217433    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 13:18:16.217564 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.217527    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 13:18:16.217706 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.217680    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 13:18:16.217823 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.217792    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 13:18:16.225426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.225407    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gbsvc"]
Sep 10 13:18:16.242944 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.242924    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:16.243021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.242926    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:16.243021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.243019    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:16.246301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.246285    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 13:18:16.246366 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.246334    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w47rx\""
Sep 10 13:18:16.246570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.246555    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 13:18:16.246793 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.246777    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 13:18:16.246860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.246847    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 13:18:16.249797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.249781    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5vlq4\""
Sep 10 13:18:16.278157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.278137    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:16.293340 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293319    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4236150-889c-4505-aade-ec24ea75d092-crio-socket\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.293442 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293344    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-registry-tls\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293442 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293359    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2bb7f1fa-465a-43b3-8884-962f45148f07-registry-certificates\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293442 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293399    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bb7f1fa-465a-43b3-8884-962f45148f07-trusted-ca\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293447    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d3bc038-4830-456d-b704-ddedf5e5d2d2-cert\") pod \"ingress-canary-mfbgb\" (UID: \"0d3bc038-4830-456d-b704-ddedf5e5d2d2\") " pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.293590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293473    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt5bb\" (UniqueName: \"kubernetes.io/projected/e4236150-889c-4505-aade-ec24ea75d092-kube-api-access-tt5bb\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.293590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293570    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9748123d-df56-462f-a50f-6bfd6d565527-metrics-tls\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.293757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293611    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2bb7f1fa-465a-43b3-8884-962f45148f07-ca-trust-extracted\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293641    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-bound-sa-token\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293701    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2bb7f1fa-465a-43b3-8884-962f45148f07-image-registry-private-configuration\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293723    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j266c\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-kube-api-access-j266c\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293745    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4236150-889c-4505-aade-ec24ea75d092-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293788    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9748123d-df56-462f-a50f-6bfd6d565527-tmp-dir\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293807    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64pb4\" (UniqueName: \"kubernetes.io/projected/9748123d-df56-462f-a50f-6bfd6d565527-kube-api-access-64pb4\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293827    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2bb7f1fa-465a-43b3-8884-962f45148f07-installation-pull-secrets\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293858    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9748123d-df56-462f-a50f-6bfd6d565527-config-volume\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293890    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4236150-889c-4505-aade-ec24ea75d092-data-volume\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293919    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4236150-889c-4505-aade-ec24ea75d092-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293922    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2bb7f1fa-465a-43b3-8884-962f45148f07-ca-trust-extracted\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.293983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.293950    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2w9c\" (UniqueName: \"kubernetes.io/projected/0d3bc038-4830-456d-b704-ddedf5e5d2d2-kube-api-access-x2w9c\") pod \"ingress-canary-mfbgb\" (UID: \"0d3bc038-4830-456d-b704-ddedf5e5d2d2\") " pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.294364 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.294202    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9748123d-df56-462f-a50f-6bfd6d565527-tmp-dir\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.294364 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.294242    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2bb7f1fa-465a-43b3-8884-962f45148f07-registry-certificates\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.294762 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.294716    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9748123d-df56-462f-a50f-6bfd6d565527-config-volume\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.295135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.295113    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bb7f1fa-465a-43b3-8884-962f45148f07-trusted-ca\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.297575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.297553    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9748123d-df56-462f-a50f-6bfd6d565527-metrics-tls\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.297740 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.297723    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d3bc038-4830-456d-b704-ddedf5e5d2d2-cert\") pod \"ingress-canary-mfbgb\" (UID: \"0d3bc038-4830-456d-b704-ddedf5e5d2d2\") " pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.297801 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.297752    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2bb7f1fa-465a-43b3-8884-962f45148f07-image-registry-private-configuration\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.297945 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.297922    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2bb7f1fa-465a-43b3-8884-962f45148f07-installation-pull-secrets\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.297980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.297933    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-registry-tls\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.303383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.303363    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64pb4\" (UniqueName: \"kubernetes.io/projected/9748123d-df56-462f-a50f-6bfd6d565527-kube-api-access-64pb4\") pod \"dns-default-wdq7m\" (UID: \"9748123d-df56-462f-a50f-6bfd6d565527\") " pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.303770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.303749    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-bound-sa-token\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.303844 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.303799    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2w9c\" (UniqueName: \"kubernetes.io/projected/0d3bc038-4830-456d-b704-ddedf5e5d2d2-kube-api-access-x2w9c\") pod \"ingress-canary-mfbgb\" (UID: \"0d3bc038-4830-456d-b704-ddedf5e5d2d2\") " pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.304008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.303987    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j266c\" (UniqueName: \"kubernetes.io/projected/2bb7f1fa-465a-43b3-8884-962f45148f07-kube-api-access-j266c\") pod \"image-registry-7fc87d7f97-48xl2\" (UID: \"2bb7f1fa-465a-43b3-8884-962f45148f07\") " pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.363217 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.363182    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:16.394430 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.394406    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4236150-889c-4505-aade-ec24ea75d092-data-volume\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.394539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.394437    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4236150-889c-4505-aade-ec24ea75d092-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.394539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.394465    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4236150-889c-4505-aade-ec24ea75d092-crio-socket\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.394539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.394510    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt5bb\" (UniqueName: \"kubernetes.io/projected/e4236150-889c-4505-aade-ec24ea75d092-kube-api-access-tt5bb\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.394642 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.394537    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4236150-889c-4505-aade-ec24ea75d092-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.394702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.394652    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e4236150-889c-4505-aade-ec24ea75d092-crio-socket\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.394793 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.394779    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4236150-889c-4505-aade-ec24ea75d092-data-volume\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.395033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.395016    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e4236150-889c-4505-aade-ec24ea75d092-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.396715 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.396693    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e4236150-889c-4505-aade-ec24ea75d092-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.402726 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.402703    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt5bb\" (UniqueName: \"kubernetes.io/projected/e4236150-889c-4505-aade-ec24ea75d092-kube-api-access-tt5bb\") pod \"insights-runtime-extractor-gbsvc\" (UID: \"e4236150-889c-4505-aade-ec24ea75d092\") " pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.410639 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.410618    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:16.426431 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.426412    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mfbgb"
Sep 10 13:18:16.432187 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.432166    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:16.522986 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.522960    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gbsvc"
Sep 10 13:18:16.690529 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.690493    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:16.706384 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.706348    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fc87d7f97-48xl2"]
Sep 10 13:18:16.711089 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:16.711069    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb7f1fa_465a_43b3_8884_962f45148f07.slice/crio-1777c426fae7455302e41d273bdf2b34a4fda3738df655921994cbbe6688b888 WatchSource:0}: Error finding container 1777c426fae7455302e41d273bdf2b34a4fda3738df655921994cbbe6688b888: Status 404 returned error can't find the container with id 1777c426fae7455302e41d273bdf2b34a4fda3738df655921994cbbe6688b888
Sep 10 13:18:16.724333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.724310    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mfbgb"]
Sep 10 13:18:16.727785 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.727761    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wdq7m"]
Sep 10 13:18:16.727953 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:16.727931    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d3bc038_4830_456d_b704_ddedf5e5d2d2.slice/crio-042136c14a470df7ced36b8e700090e4626512eaeb93851a4950a30bb360518d WatchSource:0}: Error finding container 042136c14a470df7ced36b8e700090e4626512eaeb93851a4950a30bb360518d: Status 404 returned error can't find the container with id 042136c14a470df7ced36b8e700090e4626512eaeb93851a4950a30bb360518d
Sep 10 13:18:16.742500 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:16.742470    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9748123d_df56_462f_a50f_6bfd6d565527.slice/crio-e2d5b4b3b22600dc038c69372bc4c2a0489904117c614a43ee77c9700b0af59e WatchSource:0}: Error finding container e2d5b4b3b22600dc038c69372bc4c2a0489904117c614a43ee77c9700b0af59e: Status 404 returned error can't find the container with id e2d5b4b3b22600dc038c69372bc4c2a0489904117c614a43ee77c9700b0af59e
Sep 10 13:18:16.797190 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.797166    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gbsvc"]
Sep 10 13:18:16.799705 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:16.799680    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4236150_889c_4505_aade_ec24ea75d092.slice/crio-01fd7c5a0e4e2907e24740f8776e1542f9bd86980ee9e6c14ee1a21f45e19953 WatchSource:0}: Error finding container 01fd7c5a0e4e2907e24740f8776e1542f9bd86980ee9e6c14ee1a21f45e19953: Status 404 returned error can't find the container with id 01fd7c5a0e4e2907e24740f8776e1542f9bd86980ee9e6c14ee1a21f45e19953
Sep 10 13:18:16.802226 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:16.802202    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:17.258143 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.258105    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:17.364153 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.364114    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:17.422405 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.422120    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z6jq7"
Sep 10 13:18:17.488560 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.488526    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gbsvc" event={"ID":"e4236150-889c-4505-aade-ec24ea75d092","Type":"ContainerStarted","Data":"fe3c25ed4ea5834f2ddaf5918b7f1415fa46aad8ffd173313cbf2449c98029e5"}
Sep 10 13:18:17.488734 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.488585    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gbsvc" event={"ID":"e4236150-889c-4505-aade-ec24ea75d092","Type":"ContainerStarted","Data":"01fd7c5a0e4e2907e24740f8776e1542f9bd86980ee9e6c14ee1a21f45e19953"}
Sep 10 13:18:17.489848 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.489814    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wdq7m" event={"ID":"9748123d-df56-462f-a50f-6bfd6d565527","Type":"ContainerStarted","Data":"e2d5b4b3b22600dc038c69372bc4c2a0489904117c614a43ee77c9700b0af59e"}
Sep 10 13:18:17.491086 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.491056    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mfbgb" event={"ID":"0d3bc038-4830-456d-b704-ddedf5e5d2d2","Type":"ContainerStarted","Data":"042136c14a470df7ced36b8e700090e4626512eaeb93851a4950a30bb360518d"}
Sep 10 13:18:17.492705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.492679    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2" event={"ID":"2bb7f1fa-465a-43b3-8884-962f45148f07","Type":"ContainerStarted","Data":"361c24a6d3d235c583cd4b3d71646c71d01d11d73de471882b58baac1d8a6fbb"}
Sep 10 13:18:17.492826 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.492723    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2" event={"ID":"2bb7f1fa-465a-43b3-8884-962f45148f07","Type":"ContainerStarted","Data":"1777c426fae7455302e41d273bdf2b34a4fda3738df655921994cbbe6688b888"}
Sep 10 13:18:17.492885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.492831    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:17.516811 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.516701    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2" podStartSLOduration=6.516682558 podStartE2EDuration="6.516682558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:17.516168663 +0000 UTC m=+62.727361932" watchObservedRunningTime="2026-09-10 13:18:17.516682558 +0000 UTC m=+62.727875812"
Sep 10 13:18:17.581011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.580981    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:17.681713 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.681679    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:17.794184 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:17.794156    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:18.419076 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:18.419036    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:18.483417 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:18.483354    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:18.498329 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:18.498243    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gbsvc" event={"ID":"e4236150-889c-4505-aade-ec24ea75d092","Type":"ContainerStarted","Data":"4d7a0896784de58af8996c8875a40537cd327386b7336df239e6a8a11cfc5c8f"}
Sep 10 13:18:18.719644 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:18.719620    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:19.503256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:19.503219    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wdq7m" event={"ID":"9748123d-df56-462f-a50f-6bfd6d565527","Type":"ContainerStarted","Data":"42645d9748f0d5b1919a96ca5ca947afd156fc52f7064d9c64b6e60a2c08e5f2"}
Sep 10 13:18:19.503719 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:19.503287    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wdq7m" event={"ID":"9748123d-df56-462f-a50f-6bfd6d565527","Type":"ContainerStarted","Data":"165e9314d73670de3527d2702051048a9746e411386085855b84d7980bb25538"}
Sep 10 13:18:19.503719 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:19.503477    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:19.528217 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:19.528165    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wdq7m" podStartSLOduration=1.615569545 podStartE2EDuration="3.528146085s" totalImagesPullingTime="1.91257654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:19.526747758 +0000 UTC m=+64.737941030" watchObservedRunningTime="2026-09-10 13:18:19.528146085 +0000 UTC m=+64.739339344"
Sep 10 13:18:20.508404 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.508366    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gbsvc" event={"ID":"e4236150-889c-4505-aade-ec24ea75d092","Type":"ContainerStarted","Data":"49b62d2e12b864a5380af339d2092eec11037d946b3cc4a9c9d07a1e7512ed35"}
Sep 10 13:18:20.509920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.509891    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mfbgb" event={"ID":"0d3bc038-4830-456d-b704-ddedf5e5d2d2","Type":"ContainerStarted","Data":"cf572f3585d875da4f57c77a6589ddbdf09ce5a6918a4250122b0d5908fd61aa"}
Sep 10 13:18:20.532562 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.532472    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gbsvc" podStartSLOduration=1.424928185 podStartE2EDuration="4.532457911s" totalImagesPullingTime="3.107529726s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:18:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:20.531687859 +0000 UTC m=+65.742881111" watchObservedRunningTime="2026-09-10 13:18:20.532457911 +0000 UTC m=+65.743651167"
Sep 10 13:18:20.571012 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.570974    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mfbgb" podStartSLOduration=1.277258472 podStartE2EDuration="4.570959709s" totalImagesPullingTime="3.293701237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:18:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:20.56983614 +0000 UTC m=+65.781029388" watchObservedRunningTime="2026-09-10 13:18:20.570959709 +0000 UTC m=+65.782152965"
Sep 10 13:18:20.599809 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.599782    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:20.679308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.679283    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:20.818001 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.817923    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:20.856816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.856786    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c97gc"]
Sep 10 13:18:20.859928 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.859909    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.863159 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.863140    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 13:18:20.863411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.863288    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 13:18:20.863411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.863307    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 13:18:20.863411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.863307    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 13:18:20.863411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.863396    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 13:18:20.863604 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.863432    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 13:18:20.864569 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.864554    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l4r8x\""
Sep 10 13:18:20.914050 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.914027    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:20.932501 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932478    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-wtmp\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932599 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932514    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931e55d-5857-4a71-8239-3ebd0f5a6647-metrics-client-ca\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932599 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932545    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-accelerators-collector-config\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932599 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932576    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932607    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-textfile\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932652    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:20.932790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932721    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-root\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932748    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-tls\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932773    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kpft\" (UniqueName: \"kubernetes.io/projected/4931e55d-5857-4a71-8239-3ebd0f5a6647-kube-api-access-9kpft\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.932949 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.932828    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-sys\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:20.934834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.934812    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f19d0aa1-31e9-4f08-a729-6cb6332f34c7-metrics-certs\") pod \"network-metrics-daemon-v5pcj\" (UID: \"f19d0aa1-31e9-4f08-a729-6cb6332f34c7\") " pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:20.990860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:20.990837    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:21.033376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033356    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:21.033479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033394    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-root\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033422    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-tls\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033445    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kpft\" (UniqueName: \"kubernetes.io/projected/4931e55d-5857-4a71-8239-3ebd0f5a6647-kube-api-access-9kpft\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033480    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-sys\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033484    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-root\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033539    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-wtmp\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033572    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931e55d-5857-4a71-8239-3ebd0f5a6647-metrics-client-ca\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033603    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-accelerators-collector-config\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033543    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-sys\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033786    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033828    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-textfile\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.033924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.033650    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-wtmp\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.034497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.034468    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-textfile\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.034776 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.034757    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4931e55d-5857-4a71-8239-3ebd0f5a6647-metrics-client-ca\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.034822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.034796    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-accelerators-collector-config\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.035773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.035746    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-tls\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.035856 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.035812    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fe469d7-9e50-4779-b9f0-e92eb262acb6-original-pull-secret\") pod \"global-pull-secret-syncer-8xc7t\" (UID: \"8fe469d7-9e50-4779-b9f0-e92eb262acb6\") " pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:21.035856 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.035839    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4931e55d-5857-4a71-8239-3ebd0f5a6647-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.044779 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.044757    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kpft\" (UniqueName: \"kubernetes.io/projected/4931e55d-5857-4a71-8239-3ebd0f5a6647-kube-api-access-9kpft\") pod \"node-exporter-c97gc\" (UID: \"4931e55d-5857-4a71-8239-3ebd0f5a6647\") " pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.051512 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.051493    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8xc7t"
Sep 10 13:18:21.061675 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.061650    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5pcj"
Sep 10 13:18:21.091630 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.091603    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:21.135134 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.135101    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:21.137561 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.137538    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrljz\" (UniqueName: \"kubernetes.io/projected/89a62182-da82-4aa2-9356-476fbf0d9893-kube-api-access-lrljz\") pod \"network-check-target-hzn5v\" (UID: \"89a62182-da82-4aa2-9356-476fbf0d9893\") " pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:21.169121 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.169098    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c97gc"
Sep 10 13:18:21.179710 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:21.179647    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4931e55d_5857_4a71_8239_3ebd0f5a6647.slice/crio-8c521eb117b32def0b5dac4ee377b01323521f8d95aff0d6d05838ed25b59474 WatchSource:0}: Error finding container 8c521eb117b32def0b5dac4ee377b01323521f8d95aff0d6d05838ed25b59474: Status 404 returned error can't find the container with id 8c521eb117b32def0b5dac4ee377b01323521f8d95aff0d6d05838ed25b59474
Sep 10 13:18:21.190815 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.190790    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:21.292635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.292611    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:21.331147 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.331081    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8xc7t"]
Sep 10 13:18:21.334379 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:21.334347    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe469d7_9e50_4779_b9f0_e92eb262acb6.slice/crio-c9306b756dcabbaa99dc1f1be6a51f7a1ecd0e3d09476ce98464ce1a3037cb20 WatchSource:0}: Error finding container c9306b756dcabbaa99dc1f1be6a51f7a1ecd0e3d09476ce98464ce1a3037cb20: Status 404 returned error can't find the container with id c9306b756dcabbaa99dc1f1be6a51f7a1ecd0e3d09476ce98464ce1a3037cb20
Sep 10 13:18:21.334497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.334480    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5pcj"]
Sep 10 13:18:21.337511 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:21.337489    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19d0aa1_31e9_4f08_a729_6cb6332f34c7.slice/crio-b14a4b5fa7e65624b6c05b51e5ad5b087b02c8f41a473b32a3dda674bfcf52f3 WatchSource:0}: Error finding container b14a4b5fa7e65624b6c05b51e5ad5b087b02c8f41a473b32a3dda674bfcf52f3: Status 404 returned error can't find the container with id b14a4b5fa7e65624b6c05b51e5ad5b087b02c8f41a473b32a3dda674bfcf52f3
Sep 10 13:18:21.356885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.356867    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:21.516644 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.516612    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c97gc" event={"ID":"4931e55d-5857-4a71-8239-3ebd0f5a6647","Type":"ContainerStarted","Data":"8c521eb117b32def0b5dac4ee377b01323521f8d95aff0d6d05838ed25b59474"}
Sep 10 13:18:21.517716 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.517693    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5pcj" event={"ID":"f19d0aa1-31e9-4f08-a729-6cb6332f34c7","Type":"ContainerStarted","Data":"b14a4b5fa7e65624b6c05b51e5ad5b087b02c8f41a473b32a3dda674bfcf52f3"}
Sep 10 13:18:21.518781 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.518759    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8xc7t" event={"ID":"8fe469d7-9e50-4779-b9f0-e92eb262acb6","Type":"ContainerStarted","Data":"c9306b756dcabbaa99dc1f1be6a51f7a1ecd0e3d09476ce98464ce1a3037cb20"}
Sep 10 13:18:21.601061 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.600988    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:21.624269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.624247    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hzn5v"]
Sep 10 13:18:21.627128 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:21.627107    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89a62182_da82_4aa2_9356_476fbf0d9893.slice/crio-8d023b56393b52eab9afa792a85b2c593f3c6c97c8f410f647e370813c5329a4 WatchSource:0}: Error finding container 8d023b56393b52eab9afa792a85b2c593f3c6c97c8f410f647e370813c5329a4: Status 404 returned error can't find the container with id 8d023b56393b52eab9afa792a85b2c593f3c6c97c8f410f647e370813c5329a4
Sep 10 13:18:21.700106 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.700081    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:21.799546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.799516    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:21.901649 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:21.901583    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:22.104364 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.104324    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:22.517031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.516992    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:22.524681 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.524643    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzn5v" event={"ID":"89a62182-da82-4aa2-9356-476fbf0d9893","Type":"ContainerStarted","Data":"8d023b56393b52eab9afa792a85b2c593f3c6c97c8f410f647e370813c5329a4"}
Sep 10 13:18:22.960085 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.960056    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d4584d64c-qfslj"]
Sep 10 13:18:22.965366 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.965336    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:22.968036 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.967867    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 10 13:18:22.968698 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.968491    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 10 13:18:22.969027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.968866    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vzxcg\""
Sep 10 13:18:22.969579 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.969556    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7un5b24d9279j\""
Sep 10 13:18:22.969810 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.969792    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 10 13:18:22.970146 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.969996    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 10 13:18:22.970146 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.970022    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 10 13:18:22.978409 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:22.978329    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d4584d64c-qfslj"]
Sep 10 13:18:23.054784 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.054646    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.054784 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.054707    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.054784 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.054767    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpsfn\" (UniqueName: \"kubernetes.io/projected/50bd353d-50de-4220-b161-b6b76e0611dd-kube-api-access-fpsfn\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.055002 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.054874    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-tls\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.055002 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.054923    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50bd353d-50de-4220-b161-b6b76e0611dd-metrics-client-ca\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.055002 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.054952    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-grpc-tls\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.055160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.055026    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.055160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.055053    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155627 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155550    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50bd353d-50de-4220-b161-b6b76e0611dd-metrics-client-ca\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155627 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155591    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-grpc-tls\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155627 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155626    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155658    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155706    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155736    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155763    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpsfn\" (UniqueName: \"kubernetes.io/projected/50bd353d-50de-4220-b161-b6b76e0611dd-kube-api-access-fpsfn\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.155898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.155828    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-tls\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.156440 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.156386    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50bd353d-50de-4220-b161-b6b76e0611dd-metrics-client-ca\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.159524 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.159469    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.160067 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.160016    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.160175 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.160132    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-grpc-tls\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.161018 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.160994    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-tls\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.161117 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.161020    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.161607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.161587    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50bd353d-50de-4220-b161-b6b76e0611dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.165718 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.165698    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpsfn\" (UniqueName: \"kubernetes.io/projected/50bd353d-50de-4220-b161-b6b76e0611dd-kube-api-access-fpsfn\") pod \"thanos-querier-5d4584d64c-qfslj\" (UID: \"50bd353d-50de-4220-b161-b6b76e0611dd\") " pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.319491 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.319457    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:23.366085 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.366054    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:23.460126 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.460053    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:23.531115 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.531081    2551 generic.go:356] "Generic (PLEG): container finished" podID="4931e55d-5857-4a71-8239-3ebd0f5a6647" containerID="89e560bdd34b8537d888bc6c5b4d95503541c8df62d3ad0508a64e2691dc6495" exitCode=0
Sep 10 13:18:23.531522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.531165    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c97gc" event={"ID":"4931e55d-5857-4a71-8239-3ebd0f5a6647","Type":"ContainerDied","Data":"89e560bdd34b8537d888bc6c5b4d95503541c8df62d3ad0508a64e2691dc6495"}
Sep 10 13:18:23.533913 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.533844    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5pcj" event={"ID":"f19d0aa1-31e9-4f08-a729-6cb6332f34c7","Type":"ContainerStarted","Data":"c4f551019aed0c7bece016cabacc524d4277d09445a9879eb589eddc1d3928af"}
Sep 10 13:18:23.533913 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.533899    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5pcj" event={"ID":"f19d0aa1-31e9-4f08-a729-6cb6332f34c7","Type":"ContainerStarted","Data":"f0f58f5bb1c524d9aed8fb0b13e160b9b566f96c7fbbe66c8eba007940bcc380"}
Sep 10 13:18:23.579926 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.579897    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:23.607241 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.607184    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v5pcj" podStartSLOduration=67.129974141 podStartE2EDuration="1m8.607164569s" totalImagesPullingTime="1.477190428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:23.6058427 +0000 UTC m=+68.817035957" watchObservedRunningTime="2026-09-10 13:18:23.607164569 +0000 UTC m=+68.818357826"
Sep 10 13:18:23.621311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.621280    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d4584d64c-qfslj"]
Sep 10 13:18:23.627855 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:23.627825    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50bd353d_50de_4220_b161_b6b76e0611dd.slice/crio-5057d2fdc12096700f06aeb5b38d7ee9fee9ff78e9707f09b969c05c12ab09f0 WatchSource:0}: Error finding container 5057d2fdc12096700f06aeb5b38d7ee9fee9ff78e9707f09b969c05c12ab09f0: Status 404 returned error can't find the container with id 5057d2fdc12096700f06aeb5b38d7ee9fee9ff78e9707f09b969c05c12ab09f0
Sep 10 13:18:23.743987 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.743912    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:23.846135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.846091    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:23.946143 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:23.946089    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:24.119867 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:24.119838    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:24.219187 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:24.219154    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:24.540802 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:24.540766    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c97gc" event={"ID":"4931e55d-5857-4a71-8239-3ebd0f5a6647","Type":"ContainerStarted","Data":"694c221e81673d49905a7d25626afdc0c9269dd1044681b0dc6ff4399603f88f"}
Sep 10 13:18:24.541225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:24.540828    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c97gc" event={"ID":"4931e55d-5857-4a71-8239-3ebd0f5a6647","Type":"ContainerStarted","Data":"a31fbd34689315a609a3e57e618f434cd8db0caa3c81c469ab1cf7bd7e827fc0"}
Sep 10 13:18:24.542189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:24.542160    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" event={"ID":"50bd353d-50de-4220-b161-b6b76e0611dd","Type":"ContainerStarted","Data":"5057d2fdc12096700f06aeb5b38d7ee9fee9ff78e9707f09b969c05c12ab09f0"}
Sep 10 13:18:24.569993 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:24.569950    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c97gc" podStartSLOduration=2.857826555 podStartE2EDuration="4.569934722s" totalImagesPullingTime="1.637092811s" totalInitContainerRuntime="75.015356ms" isStatefulPod=true podCreationTimestamp="2026-09-10 13:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:24.568270334 +0000 UTC m=+69.779463585" watchObservedRunningTime="2026-09-10 13:18:24.569934722 +0000 UTC m=+69.781127979"
Sep 10 13:18:25.636445 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.636409    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"]
Sep 10 13:18:25.655754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.655727    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"]
Sep 10 13:18:25.655899 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.655832    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:25.658743 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.658689    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 10 13:18:25.658862 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.658756    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vwg2c\""
Sep 10 13:18:25.665404 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.665372    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78d5799-49g92"]
Sep 10 13:18:25.666252 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.666229    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:25.678847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.678828    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.682228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.682022    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 13:18:25.682341 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.682265    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 13:18:25.683003 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.682775    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vhx9c\""
Sep 10 13:18:25.683677 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.683488    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 13:18:25.684711 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.684345    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 13:18:25.686602 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.686578    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d5799-49g92"]
Sep 10 13:18:25.688142 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.687086    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 13:18:25.688142 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.687175    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 13:18:25.688142 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.687432    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 13:18:25.705415 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.705388    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 13:18:25.753301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.753265    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:25.776263 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776194    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-serving-cert\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.776263 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776230    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-oauth-serving-cert\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.776263 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776264    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-trusted-ca-bundle\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.776471 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776335    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7n8h\" (UniqueName: \"kubernetes.io/projected/01f5a2e6-19c5-442a-a515-1bf2464dfae6-kube-api-access-h7n8h\") pod \"monitoring-plugin-85bbbbbf4-tzw2c\" (UID: \"01f5a2e6-19c5-442a-a515-1bf2464dfae6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:25.776471 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776381    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-oauth-config\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.776471 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776416    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01f5a2e6-19c5-442a-a515-1bf2464dfae6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-tzw2c\" (UID: \"01f5a2e6-19c5-442a-a515-1bf2464dfae6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:25.776609 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776495    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-service-ca\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.776609 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776554    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppc46\" (UniqueName: \"kubernetes.io/projected/351d8351-7cd2-46cd-806b-6c6157330c8b-kube-api-access-ppc46\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.776609 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.776597    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-console-config\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.877591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877551    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-service-ca\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.877792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877599    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppc46\" (UniqueName: \"kubernetes.io/projected/351d8351-7cd2-46cd-806b-6c6157330c8b-kube-api-access-ppc46\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.877792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877635    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-console-config\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.877792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877688    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-serving-cert\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.877792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877709    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-oauth-serving-cert\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.877792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877738    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-trusted-ca-bundle\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.877792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877782    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7n8h\" (UniqueName: \"kubernetes.io/projected/01f5a2e6-19c5-442a-a515-1bf2464dfae6-kube-api-access-h7n8h\") pod \"monitoring-plugin-85bbbbbf4-tzw2c\" (UID: \"01f5a2e6-19c5-442a-a515-1bf2464dfae6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:25.878144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877812    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-oauth-config\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.878144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.877844    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01f5a2e6-19c5-442a-a515-1bf2464dfae6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-tzw2c\" (UID: \"01f5a2e6-19c5-442a-a515-1bf2464dfae6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:25.878144 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:25.877964    2551 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 10 13:18:25.878144 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:25.878041    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01f5a2e6-19c5-442a-a515-1bf2464dfae6-monitoring-plugin-cert podName:01f5a2e6-19c5-442a-a515-1bf2464dfae6 nodeName:}" failed. No retries permitted until 2026-09-10 13:18:26.378021276 +0000 UTC m=+71.589214531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/01f5a2e6-19c5-442a-a515-1bf2464dfae6-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-tzw2c" (UID: "01f5a2e6-19c5-442a-a515-1bf2464dfae6") : secret "monitoring-plugin-cert" not found
Sep 10 13:18:25.878434 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.878403    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-service-ca\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.878546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.878525    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-console-config\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.878797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.878774    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-trusted-ca-bundle\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.879257 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.879230    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-oauth-serving-cert\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.880590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.880568    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-serving-cert\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.881249 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.881225    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-oauth-config\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.889363 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.889295    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppc46\" (UniqueName: \"kubernetes.io/projected/351d8351-7cd2-46cd-806b-6c6157330c8b-kube-api-access-ppc46\") pod \"console-78d5799-49g92\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") " pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:25.889472 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.889445    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7n8h\" (UniqueName: \"kubernetes.io/projected/01f5a2e6-19c5-442a-a515-1bf2464dfae6-kube-api-access-h7n8h\") pod \"monitoring-plugin-85bbbbbf4-tzw2c\" (UID: \"01f5a2e6-19c5-442a-a515-1bf2464dfae6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:25.963303 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.963254    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:25.993581 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:25.993554    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:26.382382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:26.382344    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01f5a2e6-19c5-442a-a515-1bf2464dfae6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-tzw2c\" (UID: \"01f5a2e6-19c5-442a-a515-1bf2464dfae6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:26.385278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:26.385242    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/01f5a2e6-19c5-442a-a515-1bf2464dfae6-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-tzw2c\" (UID: \"01f5a2e6-19c5-442a-a515-1bf2464dfae6\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:26.404609 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:26.404570    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:26.506966 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:26.506939    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:26.568656 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:26.568630    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:27.188515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.188441    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:18:27.192861 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.192839    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.195915 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.195892    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 13:18:27.198391 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198133    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 13:18:27.198391 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198297    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 13:18:27.198565 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198517    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 13:18:27.198730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198710    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 13:18:27.198730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198725    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 13:18:27.198890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198731    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 13:18:27.198890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198783    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 13:18:27.198890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.198728    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 13:18:27.199578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.199556    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 13:18:27.199692 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.199626    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 13:18:27.199873 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.199845    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 13:18:27.199947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.199916    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9g8hq\""
Sep 10 13:18:27.200056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.200039    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 13:18:27.200118 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.200058    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rb32rpl98m4g\""
Sep 10 13:18:27.200178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.200126    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 13:18:27.202708 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.202688    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 13:18:27.214914 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.214890    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:18:27.218029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.218009    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:27.289595 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289563    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.289767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289612    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.289767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289643    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.289767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289693    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.289767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289727    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.289767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289750    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6656k\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-kube-api-access-6656k\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289771    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289797    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289874    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289901    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289921    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289960    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.289991    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.290024    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.290052    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.290079    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.290109    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.290138    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.290167    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.290428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.290207    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393109    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393165    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393191    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393218    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393247    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393282    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393310    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393342    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393390    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393414    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393438    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393479    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393512    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393538    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393563    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393589    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.394278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393620    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.395163 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393644    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6656k\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-kube-api-access-6656k\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.395163 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393689    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.395163 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.393744    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.395163 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.394397    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.400406 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.399781    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.400406 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.400346    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.401324 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.400971    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.401324 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.400346    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.401459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.401384    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.402509 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.401814    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.405822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.405253    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.405822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.405780    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config-out\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.407387 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.407312    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.411296 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.411078    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.415120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.415076    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6656k\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-kube-api-access-6656k\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.415957 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.415254    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.416696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.416549    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.416696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.416608    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.417849 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.416932    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.418411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.418236    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.418411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.418341    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.418411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.418374    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-web-config\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.422728 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.422689    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.503881 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.503854    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:27.558145 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.558083    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8xc7t" event={"ID":"8fe469d7-9e50-4779-b9f0-e92eb262acb6","Type":"ContainerStarted","Data":"a2cac99af1467582195920e086bd4ee2ae1141a8fa80c92cb98901681139c03f"}
Sep 10 13:18:27.571488 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.571345    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" event={"ID":"50bd353d-50de-4220-b161-b6b76e0611dd","Type":"ContainerStarted","Data":"892b1a24737a245d06475500174aeda0805d7ec540d3efaa2c14cda1d0f4b615"}
Sep 10 13:18:27.571488 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.571433    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" event={"ID":"50bd353d-50de-4220-b161-b6b76e0611dd","Type":"ContainerStarted","Data":"e457597bba3c589a31d1e3e29f853ef047f63291391ecbe1d55a06e0fcc9309d"}
Sep 10 13:18:27.578923 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.577810    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hzn5v" event={"ID":"89a62182-da82-4aa2-9356-476fbf0d9893","Type":"ContainerStarted","Data":"7e9753ee00367c0877d429b4ddb94246b7a5451fbac753c783620d625a58a4d3"}
Sep 10 13:18:27.578923 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.577860    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:27.584518 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.584477    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8xc7t" podStartSLOduration=65.578680977 podStartE2EDuration="1m11.584458233s" totalImagesPullingTime="6.005777256s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:27.582985298 +0000 UTC m=+72.794178555" watchObservedRunningTime="2026-09-10 13:18:27.584458233 +0000 UTC m=+72.795651487"
Sep 10 13:18:27.604635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.604577    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hzn5v" podStartSLOduration=65.891512771 podStartE2EDuration="1m11.604557261s" totalImagesPullingTime="5.71304449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:17:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:27.604320608 +0000 UTC m=+72.815513857" watchObservedRunningTime="2026-09-10 13:18:27.604557261 +0000 UTC m=+72.815750520"
Sep 10 13:18:27.612744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.612720    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d5799-49g92"]
Sep 10 13:18:27.616170 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:27.616149    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351d8351_7cd2_46cd_806b_6c6157330c8b.slice/crio-90bde91a92f96e208e9dd9125384392e902a33a7c686378b64b506c5fe0fd9b1 WatchSource:0}: Error finding container 90bde91a92f96e208e9dd9125384392e902a33a7c686378b64b506c5fe0fd9b1: Status 404 returned error can't find the container with id 90bde91a92f96e208e9dd9125384392e902a33a7c686378b64b506c5fe0fd9b1
Sep 10 13:18:27.622917 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.622881    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"]
Sep 10 13:18:27.625165 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:27.625139    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f5a2e6_19c5_442a_a515_1bf2464dfae6.slice/crio-47e2b2a9db7fc7c8e309b0086190aec75b72673f67eaf325d26eda085dd1c11d WatchSource:0}: Error finding container 47e2b2a9db7fc7c8e309b0086190aec75b72673f67eaf325d26eda085dd1c11d: Status 404 returned error can't find the container with id 47e2b2a9db7fc7c8e309b0086190aec75b72673f67eaf325d26eda085dd1c11d
Sep 10 13:18:27.798958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:27.798929    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:18:27.801776 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:18:27.801755    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8d0eab_b779_4798_910c_2dab2c4c85fb.slice/crio-127036d80d79cb5a3f74f79d856518252864b4b168ef35b224e6155bc56c0220 WatchSource:0}: Error finding container 127036d80d79cb5a3f74f79d856518252864b4b168ef35b224e6155bc56c0220: Status 404 returned error can't find the container with id 127036d80d79cb5a3f74f79d856518252864b4b168ef35b224e6155bc56c0220
Sep 10 13:18:28.586707 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:28.586539    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" event={"ID":"50bd353d-50de-4220-b161-b6b76e0611dd","Type":"ContainerStarted","Data":"3a2142fc20f74060332321e35d468af7e664b37a5cfc9dcb378787a32c3ff3ed"}
Sep 10 13:18:28.588840 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:28.588766    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5799-49g92" event={"ID":"351d8351-7cd2-46cd-806b-6c6157330c8b","Type":"ContainerStarted","Data":"90bde91a92f96e208e9dd9125384392e902a33a7c686378b64b506c5fe0fd9b1"}
Sep 10 13:18:28.590566 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:28.590514    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c" event={"ID":"01f5a2e6-19c5-442a-a515-1bf2464dfae6","Type":"ContainerStarted","Data":"47e2b2a9db7fc7c8e309b0086190aec75b72673f67eaf325d26eda085dd1c11d"}
Sep 10 13:18:28.593095 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:28.592944    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerStarted","Data":"127036d80d79cb5a3f74f79d856518252864b4b168ef35b224e6155bc56c0220"}
Sep 10 13:18:29.512443 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:29.512410    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wdq7m"
Sep 10 13:18:29.624224 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:29.623909    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50bd353d_50de_4220_b161_b6b76e0611dd.slice/crio-4d0368a514a9daf4ce26ddc7464c7d3776cb9a155638667996c3d1a392eac6a5.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 13:18:29.737138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:29.737099    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:29.839260 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:29.839223    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:29.917484 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:29.917450    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:30.603825 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.603789    2551 generic.go:356] "Generic (PLEG): container finished" podID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f" exitCode=0
Sep 10 13:18:30.603990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.603882    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"}
Sep 10 13:18:30.607380 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.607351    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" event={"ID":"50bd353d-50de-4220-b161-b6b76e0611dd","Type":"ContainerStarted","Data":"74f7f4055f520bb6774538d8ab2c041bc910d64d10235a0f33cc09d526c97771"}
Sep 10 13:18:30.607500 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.607412    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" event={"ID":"50bd353d-50de-4220-b161-b6b76e0611dd","Type":"ContainerStarted","Data":"4d0368a514a9daf4ce26ddc7464c7d3776cb9a155638667996c3d1a392eac6a5"}
Sep 10 13:18:30.609140 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.609100    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c" event={"ID":"01f5a2e6-19c5-442a-a515-1bf2464dfae6","Type":"ContainerStarted","Data":"5a58f5ecc22ffcf9634bb9555303727869440f32d0998ed7a1d37a55268aca91"}
Sep 10 13:18:30.609356 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.609324    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:30.614815 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.614789    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c"
Sep 10 13:18:30.663785 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:30.663735    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzw2c" podStartSLOduration=3.772059953 podStartE2EDuration="5.663719347s" totalImagesPullingTime="1.891659394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:30.662852148 +0000 UTC m=+75.874045407" watchObservedRunningTime="2026-09-10 13:18:30.663719347 +0000 UTC m=+75.874912631"
Sep 10 13:18:31.616250 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:31.616212    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" event={"ID":"50bd353d-50de-4220-b161-b6b76e0611dd","Type":"ContainerStarted","Data":"e205d780b71d0977e46f01ca84f6f6313c18582f25719bcd2aec80bf7eff221a"}
Sep 10 13:18:31.616444 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:31.616366    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:31.618126 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:31.618096    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5799-49g92" event={"ID":"351d8351-7cd2-46cd-806b-6c6157330c8b","Type":"ContainerStarted","Data":"c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead"}
Sep 10 13:18:31.651888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:31.651837    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj" podStartSLOduration=3.975442089 podStartE2EDuration="9.651816368s" totalImagesPullingTime="5.676374279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:18:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:31.651084157 +0000 UTC m=+76.862277415" watchObservedRunningTime="2026-09-10 13:18:31.651816368 +0000 UTC m=+76.863009617"
Sep 10 13:18:31.677217 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:31.677166    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78d5799-49g92" podStartSLOduration=3.452560242 podStartE2EDuration="6.677148442s" totalImagesPullingTime="3.2245882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:18:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:31.676462855 +0000 UTC m=+76.887656134" watchObservedRunningTime="2026-09-10 13:18:31.677148442 +0000 UTC m=+76.888341700"
Sep 10 13:18:32.628925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:32.628894    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d4584d64c-qfslj"
Sep 10 13:18:32.849093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:32.849057    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d5799-49g92"]
Sep 10 13:18:34.640331 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:34.640298    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerStarted","Data":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"}
Sep 10 13:18:34.640705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:34.640406    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerStarted","Data":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"}
Sep 10 13:18:34.640705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:34.640452    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerStarted","Data":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"}
Sep 10 13:18:34.640705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:34.640488    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerStarted","Data":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"}
Sep 10 13:18:34.640705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:34.640524    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerStarted","Data":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"}
Sep 10 13:18:34.640705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:34.640561    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerStarted","Data":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"}
Sep 10 13:18:34.684298 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:34.684203    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.718282958 podStartE2EDuration="7.684190001s" totalImagesPullingTime="4.934279444s" totalInitContainerRuntime="31.627599ms" isStatefulPod=false podCreationTimestamp="2026-09-10 13:18:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:18:34.68111427 +0000 UTC m=+79.892307528" watchObservedRunningTime="2026-09-10 13:18:34.684190001 +0000 UTC m=+79.895383297"
Sep 10 13:18:35.785641 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:35.785585    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:35.855343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:35.855313    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:35.993926 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:35.993884    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:37.215539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:37.215504    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:37.319191 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:37.319161    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:37.504527 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:37.504467    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:18:38.503129 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:38.503021    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7fc87d7f97-48xl2"
Sep 10 13:18:38.658500 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:38.658472    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:38.869837 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:38.869806    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:39.091802 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:39.091772    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:43.913060 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:43.913028    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:44.014160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:44.014127    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:44.074920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:44.074895    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:47.702139 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:47.702103    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:47.803649 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:47.803616    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:49.438286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:49.438258    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:49.553995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:49.553960    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:49.710446 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:49.710376    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:50.582948 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:50.582916    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:50.751651 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:50.751619    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:18:58.596243 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:58.596213    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hzn5v"
Sep 10 13:18:58.651298 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:58.651267    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78d5799-49g92" podUID="351d8351-7cd2-46cd-806b-6c6157330c8b" containerName="console" containerID="cri-o://c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead" gracePeriod=15
Sep 10 13:18:58.936564 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:58.936540    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:59.029321 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029295    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-service-ca\") pod \"351d8351-7cd2-46cd-806b-6c6157330c8b\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") "
Sep 10 13:18:59.029444 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029339    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-serving-cert\") pod \"351d8351-7cd2-46cd-806b-6c6157330c8b\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") "
Sep 10 13:18:59.029444 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029372    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-oauth-config\") pod \"351d8351-7cd2-46cd-806b-6c6157330c8b\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") "
Sep 10 13:18:59.029520 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029493    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppc46\" (UniqueName: \"kubernetes.io/projected/351d8351-7cd2-46cd-806b-6c6157330c8b-kube-api-access-ppc46\") pod \"351d8351-7cd2-46cd-806b-6c6157330c8b\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") "
Sep 10 13:18:59.029570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029553    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-oauth-serving-cert\") pod \"351d8351-7cd2-46cd-806b-6c6157330c8b\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") "
Sep 10 13:18:59.029619 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029602    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-trusted-ca-bundle\") pod \"351d8351-7cd2-46cd-806b-6c6157330c8b\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") "
Sep 10 13:18:59.029685 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029630    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-console-config\") pod \"351d8351-7cd2-46cd-806b-6c6157330c8b\" (UID: \"351d8351-7cd2-46cd-806b-6c6157330c8b\") "
Sep 10 13:18:59.029749 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029681    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-service-ca" (OuterVolumeSpecName: "service-ca") pod "351d8351-7cd2-46cd-806b-6c6157330c8b" (UID: "351d8351-7cd2-46cd-806b-6c6157330c8b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:18:59.029918 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.029884    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:18:59.030041 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.030008    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "351d8351-7cd2-46cd-806b-6c6157330c8b" (UID: "351d8351-7cd2-46cd-806b-6c6157330c8b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:18:59.030104 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.030060    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "351d8351-7cd2-46cd-806b-6c6157330c8b" (UID: "351d8351-7cd2-46cd-806b-6c6157330c8b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:18:59.030157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.030121    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-console-config" (OuterVolumeSpecName: "console-config") pod "351d8351-7cd2-46cd-806b-6c6157330c8b" (UID: "351d8351-7cd2-46cd-806b-6c6157330c8b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:18:59.031609 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.031592    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "351d8351-7cd2-46cd-806b-6c6157330c8b" (UID: "351d8351-7cd2-46cd-806b-6c6157330c8b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:18:59.032032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.032012    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351d8351-7cd2-46cd-806b-6c6157330c8b-kube-api-access-ppc46" (OuterVolumeSpecName: "kube-api-access-ppc46") pod "351d8351-7cd2-46cd-806b-6c6157330c8b" (UID: "351d8351-7cd2-46cd-806b-6c6157330c8b"). InnerVolumeSpecName "kube-api-access-ppc46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:18:59.032101 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.032035    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "351d8351-7cd2-46cd-806b-6c6157330c8b" (UID: "351d8351-7cd2-46cd-806b-6c6157330c8b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:18:59.130454 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.130426    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-oauth-serving-cert\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:18:59.130454 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.130451    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-trusted-ca-bundle\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:18:59.130454 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.130460    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/351d8351-7cd2-46cd-806b-6c6157330c8b-console-config\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:18:59.130695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.130470    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-serving-cert\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:18:59.130695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.130478    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/351d8351-7cd2-46cd-806b-6c6157330c8b-console-oauth-config\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:18:59.130695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.130486    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppc46\" (UniqueName: \"kubernetes.io/projected/351d8351-7cd2-46cd-806b-6c6157330c8b-kube-api-access-ppc46\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:18:59.727851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.727809    2551 generic.go:356] "Generic (PLEG): container finished" podID="351d8351-7cd2-46cd-806b-6c6157330c8b" containerID="c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead" exitCode=0
Sep 10 13:18:59.727851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.727853    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5799-49g92"
Sep 10 13:18:59.728307 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.727885    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5799-49g92" event={"ID":"351d8351-7cd2-46cd-806b-6c6157330c8b","Type":"ContainerDied","Data":"c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead"}
Sep 10 13:18:59.728307 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.727924    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5799-49g92" event={"ID":"351d8351-7cd2-46cd-806b-6c6157330c8b","Type":"ContainerDied","Data":"90bde91a92f96e208e9dd9125384392e902a33a7c686378b64b506c5fe0fd9b1"}
Sep 10 13:18:59.728307 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.727944    2551 scope.go:117] "RemoveContainer" containerID="c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead"
Sep 10 13:18:59.736032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.736016    2551 scope.go:117] "RemoveContainer" containerID="c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead"
Sep 10 13:18:59.736319 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:18:59.736290    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead\": container with ID starting with c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead not found: ID does not exist" containerID="c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead"
Sep 10 13:18:59.736419 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.736321    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead"} err="failed to get container status \"c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead\": rpc error: code = NotFound desc = could not find container \"c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead\": container with ID starting with c2f492d76161627f876a84eb2efa70e66d3d431d3eb4bb45017860d0af689ead not found: ID does not exist"
Sep 10 13:18:59.753092 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.753066    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d5799-49g92"]
Sep 10 13:18:59.757437 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:18:59.757414    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78d5799-49g92"]
Sep 10 13:19:00.622303 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:00.622272    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:01.246718 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:01.246683    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="351d8351-7cd2-46cd-806b-6c6157330c8b" path="/var/lib/kubelet/pods/351d8351-7cd2-46cd-806b-6c6157330c8b/volumes"
Sep 10 13:19:05.604178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:05.604149    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:05.805281 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:05.805248    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:06.760125 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:06.760093    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:27.505113 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:27.505077    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:27.524233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:27.523944    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:27.837659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:27.837631    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:37.165289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:37.165234    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:37.250613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:37.250591    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:45.920988 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:45.920956    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:19:45.921575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:45.921375    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="prometheus" containerID="cri-o://e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c" gracePeriod=600
Sep 10 13:19:45.921575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:45.921457    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-thanos" containerID="cri-o://534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c" gracePeriod=600
Sep 10 13:19:45.921575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:45.921474    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy" containerID="cri-o://4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be" gracePeriod=600
Sep 10 13:19:45.921575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:45.921521    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="thanos-sidecar" containerID="cri-o://a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116" gracePeriod=600
Sep 10 13:19:45.921817 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:45.921451    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-web" containerID="cri-o://60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8" gracePeriod=600
Sep 10 13:19:45.921817 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:45.921728    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="config-reloader" containerID="cri-o://88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40" gracePeriod=600
Sep 10 13:19:46.151148 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.151120    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:46.260644 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260585    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config-out\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260644 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260619    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260645    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-tls\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260695    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6656k\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-kube-api-access-6656k\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260731    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-thanos-prometheus-http-client-file\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260758    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-1\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260786    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-metrics-client-ca\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260820    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-web-config\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.260863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260848    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-metrics-client-certs\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260871    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-2\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260896    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-kubelet-serving-ca-bundle\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260924    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-kube-rbac-proxy\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.260968    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-0\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261001    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-db\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261033    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-grpc-tls\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261069    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261101    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261137    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-serving-certs-ca-bundle\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261167    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-trusted-ca-bundle\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261202    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-tls-assets\") pod \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\" (UID: \"ea8d0eab-b779-4798-910c-2dab2c4c85fb\") "
Sep 10 13:19:46.261854 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261759    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:19:46.262048 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.261892    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:19:46.262554 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.262364    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:19:46.262653 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.262614    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:19:46.264369 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.264314    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:19:46.264591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.264486    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:19:46.264906 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.264879    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:19:46.265113 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.265085    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:19:46.265183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.265141    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config" (OuterVolumeSpecName: "config") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.265586 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.265560    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.265851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.265761    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.265851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.265784    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config-out" (OuterVolumeSpecName: "config-out") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:19:46.266089 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.266048    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.266192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.266095    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-kube-api-access-6656k" (OuterVolumeSpecName: "kube-api-access-6656k") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "kube-api-access-6656k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:19:46.266192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.266139    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.266320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.266218    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.266320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.266302    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.266428 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.266386    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.267002 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.266982    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:19:46.275077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.275051    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-web-config" (OuterVolumeSpecName: "web-config") pod "ea8d0eab-b779-4798-910c-2dab2c4c85fb" (UID: "ea8d0eab-b779-4798-910c-2dab2c4c85fb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:19:46.362111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362076    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362108    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-db\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362118    2551 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-grpc-tls\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362128    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362137    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362146    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362154    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362163    2551 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-tls-assets\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362171    2551 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config-out\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362180    2551 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-config\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362187    2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362195    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6656k\" (UniqueName: \"kubernetes.io/projected/ea8d0eab-b779-4798-910c-2dab2c4c85fb-kube-api-access-6656k\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362203    2551 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362212    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362224    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-metrics-client-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362233    2551 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-web-config\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362240    2551 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-metrics-client-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362248    2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362256    2551 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea8d0eab-b779-4798-910c-2dab2c4c85fb-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.362287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.362265    2551 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea8d0eab-b779-4798-910c-2dab2c4c85fb-secret-kube-rbac-proxy\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:19:46.888056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888025    2551 generic.go:356] "Generic (PLEG): container finished" podID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c" exitCode=0
Sep 10 13:19:46.888056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888048    2551 generic.go:356] "Generic (PLEG): container finished" podID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be" exitCode=0
Sep 10 13:19:46.888056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888054    2551 generic.go:356] "Generic (PLEG): container finished" podID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8" exitCode=0
Sep 10 13:19:46.888056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888059    2551 generic.go:356] "Generic (PLEG): container finished" podID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116" exitCode=0
Sep 10 13:19:46.888056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888064    2551 generic.go:356] "Generic (PLEG): container finished" podID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40" exitCode=0
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888070    2551 generic.go:356] "Generic (PLEG): container finished" podID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c" exitCode=0
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888088    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"}
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888101    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888143    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"}
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888170    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"}
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888192    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"}
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888211    2551 scope.go:117] "RemoveContainer" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888226    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"}
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888253    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"}
Sep 10 13:19:46.888335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.888282    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ea8d0eab-b779-4798-910c-2dab2c4c85fb","Type":"ContainerDied","Data":"127036d80d79cb5a3f74f79d856518252864b4b168ef35b224e6155bc56c0220"}
Sep 10 13:19:46.895280 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.895265    2551 scope.go:117] "RemoveContainer" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.901646 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.901631    2551 scope.go:117] "RemoveContainer" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.908356 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.908341    2551 scope.go:117] "RemoveContainer" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.914234 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.914218    2551 scope.go:117] "RemoveContainer" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.920043 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.920029    2551 scope.go:117] "RemoveContainer" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.922616 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.922595    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:19:46.927219 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.927196    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:19:46.927819 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.927655    2551 scope.go:117] "RemoveContainer" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.945322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.945307    2551 scope.go:117] "RemoveContainer" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.945606 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:19:46.945586    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": container with ID starting with 534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c not found: ID does not exist" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.945700 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.945616    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"} err="failed to get container status \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": rpc error: code = NotFound desc = could not find container \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": container with ID starting with 534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c not found: ID does not exist"
Sep 10 13:19:46.945700 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.945638    2551 scope.go:117] "RemoveContainer" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.946042 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:19:46.946024    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": container with ID starting with 4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be not found: ID does not exist" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.946102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946047    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"} err="failed to get container status \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": rpc error: code = NotFound desc = could not find container \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": container with ID starting with 4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be not found: ID does not exist"
Sep 10 13:19:46.946102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946063    2551 scope.go:117] "RemoveContainer" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.946299 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:19:46.946281    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": container with ID starting with 60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8 not found: ID does not exist" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.946358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946308    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"} err="failed to get container status \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": rpc error: code = NotFound desc = could not find container \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": container with ID starting with 60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8 not found: ID does not exist"
Sep 10 13:19:46.946358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946331    2551 scope.go:117] "RemoveContainer" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.946566 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:19:46.946551    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": container with ID starting with a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116 not found: ID does not exist" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.946621 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946571    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"} err="failed to get container status \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": rpc error: code = NotFound desc = could not find container \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": container with ID starting with a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116 not found: ID does not exist"
Sep 10 13:19:46.946621 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946583    2551 scope.go:117] "RemoveContainer" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.946813 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:19:46.946797    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": container with ID starting with 88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40 not found: ID does not exist" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.946881 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946820    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"} err="failed to get container status \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": rpc error: code = NotFound desc = could not find container \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": container with ID starting with 88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40 not found: ID does not exist"
Sep 10 13:19:46.946881 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.946838    2551 scope.go:117] "RemoveContainer" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.947098 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:19:46.947082    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": container with ID starting with e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c not found: ID does not exist" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.947144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947102    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"} err="failed to get container status \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": rpc error: code = NotFound desc = could not find container \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": container with ID starting with e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c not found: ID does not exist"
Sep 10 13:19:46.947144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947116    2551 scope.go:117] "RemoveContainer" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.947392 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:19:46.947375    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": container with ID starting with 36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f not found: ID does not exist" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.947453 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947400    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"} err="failed to get container status \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": rpc error: code = NotFound desc = could not find container \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": container with ID starting with 36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f not found: ID does not exist"
Sep 10 13:19:46.947453 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947416    2551 scope.go:117] "RemoveContainer" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.947629 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947613    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"} err="failed to get container status \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": rpc error: code = NotFound desc = could not find container \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": container with ID starting with 534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c not found: ID does not exist"
Sep 10 13:19:46.947700 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947628    2551 scope.go:117] "RemoveContainer" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.947894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947876    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"} err="failed to get container status \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": rpc error: code = NotFound desc = could not find container \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": container with ID starting with 4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be not found: ID does not exist"
Sep 10 13:19:46.947933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.947896    2551 scope.go:117] "RemoveContainer" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.948134 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948114    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"} err="failed to get container status \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": rpc error: code = NotFound desc = could not find container \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": container with ID starting with 60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8 not found: ID does not exist"
Sep 10 13:19:46.948202 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948135    2551 scope.go:117] "RemoveContainer" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.948410 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948392    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"} err="failed to get container status \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": rpc error: code = NotFound desc = could not find container \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": container with ID starting with a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116 not found: ID does not exist"
Sep 10 13:19:46.948460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948411    2551 scope.go:117] "RemoveContainer" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.948638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948618    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"} err="failed to get container status \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": rpc error: code = NotFound desc = could not find container \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": container with ID starting with 88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40 not found: ID does not exist"
Sep 10 13:19:46.948728 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948640    2551 scope.go:117] "RemoveContainer" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.948878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948862    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"} err="failed to get container status \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": rpc error: code = NotFound desc = could not find container \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": container with ID starting with e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c not found: ID does not exist"
Sep 10 13:19:46.948945 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.948879    2551 scope.go:117] "RemoveContainer" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.949135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949118    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"} err="failed to get container status \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": rpc error: code = NotFound desc = could not find container \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": container with ID starting with 36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f not found: ID does not exist"
Sep 10 13:19:46.949175 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949136    2551 scope.go:117] "RemoveContainer" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.949380 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949345    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"} err="failed to get container status \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": rpc error: code = NotFound desc = could not find container \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": container with ID starting with 534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c not found: ID does not exist"
Sep 10 13:19:46.949380 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949372    2551 scope.go:117] "RemoveContainer" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.949615 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949597    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"} err="failed to get container status \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": rpc error: code = NotFound desc = could not find container \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": container with ID starting with 4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be not found: ID does not exist"
Sep 10 13:19:46.949702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949615    2551 scope.go:117] "RemoveContainer" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.949861 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949840    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"} err="failed to get container status \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": rpc error: code = NotFound desc = could not find container \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": container with ID starting with 60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8 not found: ID does not exist"
Sep 10 13:19:46.949964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.949862    2551 scope.go:117] "RemoveContainer" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.950223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950186    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"} err="failed to get container status \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": rpc error: code = NotFound desc = could not find container \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": container with ID starting with a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116 not found: ID does not exist"
Sep 10 13:19:46.950223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950217    2551 scope.go:117] "RemoveContainer" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.950457 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950433    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"} err="failed to get container status \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": rpc error: code = NotFound desc = could not find container \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": container with ID starting with 88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40 not found: ID does not exist"
Sep 10 13:19:46.950503 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950458    2551 scope.go:117] "RemoveContainer" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.950715 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950693    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"} err="failed to get container status \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": rpc error: code = NotFound desc = could not find container \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": container with ID starting with e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c not found: ID does not exist"
Sep 10 13:19:46.950715 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950714    2551 scope.go:117] "RemoveContainer" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.950918 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950903    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"} err="failed to get container status \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": rpc error: code = NotFound desc = could not find container \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": container with ID starting with 36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f not found: ID does not exist"
Sep 10 13:19:46.950964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.950918    2551 scope.go:117] "RemoveContainer" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.951111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951096    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"} err="failed to get container status \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": rpc error: code = NotFound desc = could not find container \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": container with ID starting with 534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c not found: ID does not exist"
Sep 10 13:19:46.951111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951110    2551 scope.go:117] "RemoveContainer" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.951316 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951300    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"} err="failed to get container status \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": rpc error: code = NotFound desc = could not find container \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": container with ID starting with 4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be not found: ID does not exist"
Sep 10 13:19:46.951357 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951317    2551 scope.go:117] "RemoveContainer" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.951540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951522    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"} err="failed to get container status \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": rpc error: code = NotFound desc = could not find container \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": container with ID starting with 60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8 not found: ID does not exist"
Sep 10 13:19:46.951583 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951540    2551 scope.go:117] "RemoveContainer" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.951747 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951732    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"} err="failed to get container status \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": rpc error: code = NotFound desc = could not find container \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": container with ID starting with a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116 not found: ID does not exist"
Sep 10 13:19:46.951799 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951747    2551 scope.go:117] "RemoveContainer" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.951970 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951954    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"} err="failed to get container status \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": rpc error: code = NotFound desc = could not find container \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": container with ID starting with 88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40 not found: ID does not exist"
Sep 10 13:19:46.952007 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.951970    2551 scope.go:117] "RemoveContainer" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.952169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952153    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"} err="failed to get container status \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": rpc error: code = NotFound desc = could not find container \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": container with ID starting with e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c not found: ID does not exist"
Sep 10 13:19:46.952211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952169    2551 scope.go:117] "RemoveContainer" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.952367 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952351    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"} err="failed to get container status \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": rpc error: code = NotFound desc = could not find container \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": container with ID starting with 36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f not found: ID does not exist"
Sep 10 13:19:46.952407 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952366    2551 scope.go:117] "RemoveContainer" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.952577 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952561    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"} err="failed to get container status \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": rpc error: code = NotFound desc = could not find container \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": container with ID starting with 534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c not found: ID does not exist"
Sep 10 13:19:46.952620 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952578    2551 scope.go:117] "RemoveContainer" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.952791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952776    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"} err="failed to get container status \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": rpc error: code = NotFound desc = could not find container \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": container with ID starting with 4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be not found: ID does not exist"
Sep 10 13:19:46.952791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952791    2551 scope.go:117] "RemoveContainer" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.952976 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952961    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"} err="failed to get container status \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": rpc error: code = NotFound desc = could not find container \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": container with ID starting with 60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8 not found: ID does not exist"
Sep 10 13:19:46.952976 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.952976    2551 scope.go:117] "RemoveContainer" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.953157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953138    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"} err="failed to get container status \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": rpc error: code = NotFound desc = could not find container \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": container with ID starting with a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116 not found: ID does not exist"
Sep 10 13:19:46.953198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953157    2551 scope.go:117] "RemoveContainer" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.953344 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953329    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"} err="failed to get container status \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": rpc error: code = NotFound desc = could not find container \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": container with ID starting with 88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40 not found: ID does not exist"
Sep 10 13:19:46.953386 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953344    2551 scope.go:117] "RemoveContainer" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.953518 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953503    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"} err="failed to get container status \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": rpc error: code = NotFound desc = could not find container \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": container with ID starting with e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c not found: ID does not exist"
Sep 10 13:19:46.953557 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953518    2551 scope.go:117] "RemoveContainer" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.953748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953732    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"} err="failed to get container status \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": rpc error: code = NotFound desc = could not find container \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": container with ID starting with 36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f not found: ID does not exist"
Sep 10 13:19:46.953794 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953747    2551 scope.go:117] "RemoveContainer" containerID="534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"
Sep 10 13:19:46.953944 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953929    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c"} err="failed to get container status \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": rpc error: code = NotFound desc = could not find container \"534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c\": container with ID starting with 534b246ae441372aef8215125a4d12d0c3e31018f8aa2c7240492c37c776a26c not found: ID does not exist"
Sep 10 13:19:46.953944 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.953943    2551 scope.go:117] "RemoveContainer" containerID="4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"
Sep 10 13:19:46.954122 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954106    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be"} err="failed to get container status \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": rpc error: code = NotFound desc = could not find container \"4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be\": container with ID starting with 4bb2c74d5f32a342497c11a1a308f0a57c44b5cf66f266ba05837732043327be not found: ID does not exist"
Sep 10 13:19:46.954122 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954121    2551 scope.go:117] "RemoveContainer" containerID="60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"
Sep 10 13:19:46.954339 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954323    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8"} err="failed to get container status \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": rpc error: code = NotFound desc = could not find container \"60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8\": container with ID starting with 60d7f85a1f2cfca4c10f7e8e0b22b28302721c419421aae3c4673a4920bc10b8 not found: ID does not exist"
Sep 10 13:19:46.954380 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954339    2551 scope.go:117] "RemoveContainer" containerID="a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"
Sep 10 13:19:46.954527 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954511    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116"} err="failed to get container status \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": rpc error: code = NotFound desc = could not find container \"a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116\": container with ID starting with a3cffb317885eb7f32bb6ed067c811487c2262218c819cb7f0767d7988c5c116 not found: ID does not exist"
Sep 10 13:19:46.954567 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954527    2551 scope.go:117] "RemoveContainer" containerID="88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"
Sep 10 13:19:46.954754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954738    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40"} err="failed to get container status \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": rpc error: code = NotFound desc = could not find container \"88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40\": container with ID starting with 88b9df1ddc61f6ec6dc158835e00fd8e8276fbe4516cbc90b19a94415ae68d40 not found: ID does not exist"
Sep 10 13:19:46.954804 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954754    2551 scope.go:117] "RemoveContainer" containerID="e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"
Sep 10 13:19:46.954980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954963    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c"} err="failed to get container status \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": rpc error: code = NotFound desc = could not find container \"e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c\": container with ID starting with e3d2dd55d1791443a2bcb64480d91e1ea9d96fa7528f4ce4931b605cbe9b740c not found: ID does not exist"
Sep 10 13:19:46.954980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.954979    2551 scope.go:117] "RemoveContainer" containerID="36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"
Sep 10 13:19:46.955154 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.955139    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f"} err="failed to get container status \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": rpc error: code = NotFound desc = could not find container \"36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f\": container with ID starting with 36b44e185e97371f3e2323077cb8f19e5c85bd8f9277744f705eb6ed9d637d6f not found: ID does not exist"
Sep 10 13:19:46.961642 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961621    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:19:46.961865 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961852    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="thanos-sidecar"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961867    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="thanos-sidecar"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961876    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-thanos"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961881    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-thanos"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961891    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="prometheus"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961897    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="prometheus"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961908    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="init-config-reloader"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961913    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="init-config-reloader"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961919    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351d8351-7cd2-46cd-806b-6c6157330c8b" containerName="console"
Sep 10 13:19:46.961924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961924    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="351d8351-7cd2-46cd-806b-6c6157330c8b" containerName="console"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961935    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961940    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961946    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="config-reloader"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961951    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="config-reloader"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961957    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-web"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.961962    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-web"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.962000    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="thanos-sidecar"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.962007    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-web"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.962013    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.962018    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="kube-rbac-proxy-thanos"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.962024    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="prometheus"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.962028    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" containerName="config-reloader"
Sep 10 13:19:46.962197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.962034    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="351d8351-7cd2-46cd-806b-6c6157330c8b" containerName="console"
Sep 10 13:19:46.966378 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.966362    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:46.975720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.975703    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 10 13:19:46.976959 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.976940    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 10 13:19:46.977008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.976986    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 10 13:19:46.977056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977041    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 10 13:19:46.977209 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977194    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 10 13:19:46.977542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977523    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 10 13:19:46.977625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977608    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 13:19:46.977702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977689    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 10 13:19:46.977702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977660    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9g8hq\""
Sep 10 13:19:46.977790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977706    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 10 13:19:46.977995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.977980    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 10 13:19:46.978162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.978148    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 10 13:19:46.978196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.978159    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 10 13:19:46.978231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.978149    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 10 13:19:46.979626 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.979611    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6rb32rpl98m4g\""
Sep 10 13:19:46.982841 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.982823    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 10 13:19:46.985401 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.985381    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 10 13:19:46.995304 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:46.995285    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:19:47.067770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067743    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067770    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067812    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067826    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhxgt\" (UniqueName: \"kubernetes.io/projected/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-kube-api-access-rhxgt\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067841    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-config\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067860    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067876    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.067925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067894    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067939    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067967    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.067992    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068016    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068034    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068050    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-config-out\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068071    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-web-config\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068120    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068138    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068158    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.068196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.068179    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169131    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169162    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169184    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169362 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169216    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169362 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169327    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhxgt\" (UniqueName: \"kubernetes.io/projected/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-kube-api-access-rhxgt\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169362 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169356    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-config\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169504 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169395    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169504 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169418    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169504 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169456    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169504 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169487    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169518    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169551    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169578    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169606    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169633    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-config-out\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169657    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-web-config\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169724    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169752    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169779    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169801    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169905    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169954    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.169989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.169988    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.170955 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.170649    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.172482 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.172396    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.172589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.172484    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.172589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.172571    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-config-out\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.172892 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.172735    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-config\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.172892 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.172762    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.172892 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.172762    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.173093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.172986    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.173276 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.173208    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-web-config\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.178294 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.173629    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.178294 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.173728    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.178294 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.174171    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.179448 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.179423    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.179542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.179465    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.179542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.179508    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.180372 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.180350    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.186570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.186549    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhxgt\" (UniqueName: \"kubernetes.io/projected/223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb-kube-api-access-rhxgt\") pod \"prometheus-k8s-0\" (UID: \"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.247383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.247357    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8d0eab-b779-4798-910c-2dab2c4c85fb" path="/var/lib/kubelet/pods/ea8d0eab-b779-4798-910c-2dab2c4c85fb/volumes"
Sep 10 13:19:47.274540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.274521    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:47.568264 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.568242    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 10 13:19:47.570313 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:19:47.570278    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223e3a49_1ab8_4b3d_a7aa_1b97d92aeadb.slice/crio-8514d1bb9c579eea72d344ec7c5dc9f6e96ed8a6e7667e8395d9d052cedf002a WatchSource:0}: Error finding container 8514d1bb9c579eea72d344ec7c5dc9f6e96ed8a6e7667e8395d9d052cedf002a: Status 404 returned error can't find the container with id 8514d1bb9c579eea72d344ec7c5dc9f6e96ed8a6e7667e8395d9d052cedf002a
Sep 10 13:19:47.894354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.894285    2551 generic.go:356] "Generic (PLEG): container finished" podID="223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb" containerID="7d23e9c6c22bf96b1b101dc3e7c24c0709cd113857483c1d422b3a56de8bb3a3" exitCode=0
Sep 10 13:19:47.894354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.894321    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerDied","Data":"7d23e9c6c22bf96b1b101dc3e7c24c0709cd113857483c1d422b3a56de8bb3a3"}
Sep 10 13:19:47.894504 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:47.894370    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerStarted","Data":"8514d1bb9c579eea72d344ec7c5dc9f6e96ed8a6e7667e8395d9d052cedf002a"}
Sep 10 13:19:48.901463 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:48.901425    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerStarted","Data":"ec349fdf389eae7217be939fba5ef27ab918ba4deb3fc1a7ae692720bee354fe"}
Sep 10 13:19:48.901847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:48.901505    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerStarted","Data":"96cc4a30adcf5dcf85fe31eeb47ea18bc4d843d888a88b969655a4dfe0dade53"}
Sep 10 13:19:48.901847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:48.901545    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerStarted","Data":"6e6b387b3b4775cc12af5369c737d655c42906ac4b7428c5b723068bd8b91e56"}
Sep 10 13:19:48.901847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:48.901581    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerStarted","Data":"9254934a0620c29e1fc7812de2f7816913a5f12507d5e13a933f088cec099e28"}
Sep 10 13:19:48.901847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:48.901617    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerStarted","Data":"0823757f7efab23ed692a051227637dff43c2d7ab5aef5e447ebe4e79dc60647"}
Sep 10 13:19:48.901847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:48.901658    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb","Type":"ContainerStarted","Data":"c3177680c35c615131fac2499a6afb8706ef9ef7ba92f45e1eb36cd02b2a383e"}
Sep 10 13:19:48.941972 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:48.941926    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.919235674 podStartE2EDuration="2.941908285s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.672611ms" isStatefulPod=false podCreationTimestamp="2026-09-10 13:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:19:48.941551648 +0000 UTC m=+154.152744904" watchObservedRunningTime="2026-09-10 13:19:48.941908285 +0000 UTC m=+154.153101542"
Sep 10 13:19:49.063067 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:49.063036    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:49.136458 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:49.136431    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:52.275232 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:52.275201    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:19:58.135189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:58.135159    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:58.231962 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:58.231932    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:19:58.975459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:19:58.975424    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:00.181021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:00.180989    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:00.269950 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:00.269923    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:04.163132 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:04.163100    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:10.204748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:10.204660    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:10.304924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:10.304888    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:10.407548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:10.407523    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:25.068686 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:25.068631    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:25.142409 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:25.142381    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:40.529285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:40.529252    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:40.630092 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:40.630062    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:20:47.274900 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:47.274865    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:20:47.289964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:47.289939    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:20:48.123562 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:20:48.123536    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 10 13:21:08.857212 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:08.857178    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:09.201201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:09.201120    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:13.074640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:13.074608    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:13.187707 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:13.187676    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:14.219719 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:14.219690    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:14.302505 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:14.302477    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:14.720102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:14.720074    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:14.812497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:14.812460    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:16.396354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:16.396321    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:16.634753 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:16.634726    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:19.847277 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:19.847245    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:19.906758 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:19.906731    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:25.250696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:25.250650    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:37.043992 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:37.043959    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:37.149403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:37.149373    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:37.846938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:37.846861    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:37.945073 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:37.945050    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:38.505429 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.505399    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"]
Sep 10 13:21:38.507987 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.507970    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:38.520971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.520955    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 13:21:38.522079 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.522066    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 13:21:38.522123 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.522079    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cdfm7\""
Sep 10 13:21:38.592450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.592426    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"]
Sep 10 13:21:38.600271 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.600244    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxcn6\" (UniqueName: \"kubernetes.io/projected/70f64087-b71f-4387-b2aa-2b13186e318b-kube-api-access-wxcn6\") pod \"openshift-lws-operator-fd8ccff4c-qt4jg\" (UID: \"70f64087-b71f-4387-b2aa-2b13186e318b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:38.600367 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.600310    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70f64087-b71f-4387-b2aa-2b13186e318b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qt4jg\" (UID: \"70f64087-b71f-4387-b2aa-2b13186e318b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:38.701207 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.701183    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70f64087-b71f-4387-b2aa-2b13186e318b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qt4jg\" (UID: \"70f64087-b71f-4387-b2aa-2b13186e318b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:38.701309 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.701218    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxcn6\" (UniqueName: \"kubernetes.io/projected/70f64087-b71f-4387-b2aa-2b13186e318b-kube-api-access-wxcn6\") pod \"openshift-lws-operator-fd8ccff4c-qt4jg\" (UID: \"70f64087-b71f-4387-b2aa-2b13186e318b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:38.701549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.701530    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70f64087-b71f-4387-b2aa-2b13186e318b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-qt4jg\" (UID: \"70f64087-b71f-4387-b2aa-2b13186e318b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:38.711000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.710979    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxcn6\" (UniqueName: \"kubernetes.io/projected/70f64087-b71f-4387-b2aa-2b13186e318b-kube-api-access-wxcn6\") pod \"openshift-lws-operator-fd8ccff4c-qt4jg\" (UID: \"70f64087-b71f-4387-b2aa-2b13186e318b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:38.827948 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:38.827924    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"
Sep 10 13:21:39.101426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:39.101360    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg"]
Sep 10 13:21:39.104657 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:21:39.104634    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70f64087_b71f_4387_b2aa_2b13186e318b.slice/crio-e9f1a414265484ee9f9dc84bc1c22d5385a575a731a769d549cb1a95af7d24b2 WatchSource:0}: Error finding container e9f1a414265484ee9f9dc84bc1c22d5385a575a731a769d549cb1a95af7d24b2: Status 404 returned error can't find the container with id e9f1a414265484ee9f9dc84bc1c22d5385a575a731a769d549cb1a95af7d24b2
Sep 10 13:21:39.277544 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:39.277511    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg" event={"ID":"70f64087-b71f-4387-b2aa-2b13186e318b","Type":"ContainerStarted","Data":"e9f1a414265484ee9f9dc84bc1c22d5385a575a731a769d549cb1a95af7d24b2"}
Sep 10 13:21:42.291439 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:42.291409    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg" event={"ID":"70f64087-b71f-4387-b2aa-2b13186e318b","Type":"ContainerStarted","Data":"89dd1cd3ce931cb862b68ca2aa89b83e08c6fa5a909dcb156f2f4a9fef52f8ba"}
Sep 10 13:21:42.322820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:42.322776    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-qt4jg" podStartSLOduration=2.058307625 podStartE2EDuration="4.322761941s" totalImagesPullingTime="2.264454316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:21:42.320634022 +0000 UTC m=+267.531827280" watchObservedRunningTime="2026-09-10 13:21:42.322761941 +0000 UTC m=+267.533955198"
Sep 10 13:21:43.211294 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:43.211264    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:43.308310 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:43.308280    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:45.081546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:45.081510    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:57.402173 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.402140    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:57.528571 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.528543    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"]
Sep 10 13:21:57.535000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.534981    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.538549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.538526    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 13:21:57.539328 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.539313    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 13:21:57.540331 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.540315    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 13:21:57.541556 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.541542    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jw76b\""
Sep 10 13:21:57.552833 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.552814    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"]
Sep 10 13:21:57.632103 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.632075    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8c182f4d-0e12-4262-935a-75ee27cd3de0-manager-config\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.632194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.632107    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c182f4d-0e12-4262-935a-75ee27cd3de0-metrics-cert\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.632194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.632123    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wvp7\" (UniqueName: \"kubernetes.io/projected/8c182f4d-0e12-4262-935a-75ee27cd3de0-kube-api-access-2wvp7\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.632194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.632139    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c182f4d-0e12-4262-935a-75ee27cd3de0-cert\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.703650 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.703600    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:21:57.732845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.732821    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8c182f4d-0e12-4262-935a-75ee27cd3de0-manager-config\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.732920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.732850    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c182f4d-0e12-4262-935a-75ee27cd3de0-metrics-cert\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.732920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.732868    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wvp7\" (UniqueName: \"kubernetes.io/projected/8c182f4d-0e12-4262-935a-75ee27cd3de0-kube-api-access-2wvp7\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.732920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.732889    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c182f4d-0e12-4262-935a-75ee27cd3de0-cert\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.733460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.733439    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8c182f4d-0e12-4262-935a-75ee27cd3de0-manager-config\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.735312 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.735294    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c182f4d-0e12-4262-935a-75ee27cd3de0-metrics-cert\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.735383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.735296    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8c182f4d-0e12-4262-935a-75ee27cd3de0-cert\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.754929 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.754903    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wvp7\" (UniqueName: \"kubernetes.io/projected/8c182f4d-0e12-4262-935a-75ee27cd3de0-kube-api-access-2wvp7\") pod \"lws-controller-manager-78dbbc87c8-tgcj2\" (UID: \"8c182f4d-0e12-4262-935a-75ee27cd3de0\") " pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:57.843493 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:57.843467    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:21:58.174827 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:58.174756    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"]
Sep 10 13:21:58.177295 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:21:58.177266    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c182f4d_0e12_4262_935a_75ee27cd3de0.slice/crio-a634746736df5dc1190e88c5da33717c3ad7422f91e962750f8e8cf6b2c1c11b WatchSource:0}: Error finding container a634746736df5dc1190e88c5da33717c3ad7422f91e962750f8e8cf6b2c1c11b: Status 404 returned error can't find the container with id a634746736df5dc1190e88c5da33717c3ad7422f91e962750f8e8cf6b2c1c11b
Sep 10 13:21:58.347602 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:21:58.347570    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2" event={"ID":"8c182f4d-0e12-4262-935a-75ee27cd3de0","Type":"ContainerStarted","Data":"a634746736df5dc1190e88c5da33717c3ad7422f91e962750f8e8cf6b2c1c11b"}
Sep 10 13:22:00.292477 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:00.292444    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:00.356910 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:00.356880    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2" event={"ID":"8c182f4d-0e12-4262-935a-75ee27cd3de0","Type":"ContainerStarted","Data":"00cee7e969ca27f0fa942311423d8c8651d14a911278cdff9d2f4931091950c8"}
Sep 10 13:22:00.357031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:00.356955    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:22:00.407031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:00.406987    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2" podStartSLOduration=2.002432329 podStartE2EDuration="3.406971414s" totalImagesPullingTime="1.404539085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:21:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:00.404304183 +0000 UTC m=+285.615497440" watchObservedRunningTime="2026-09-10 13:22:00.406971414 +0000 UTC m=+285.618164671"
Sep 10 13:22:00.470638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:00.470615    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:00.789274 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:00.789243    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:02.279439 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.279393    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"]
Sep 10 13:22:02.285000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.284980    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.289203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.289165    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 13:22:02.289854 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.289832    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 13:22:02.289946 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.289871    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lrcqt\""
Sep 10 13:22:02.297045 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.297002    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 13:22:02.303956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.303940    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 13:22:02.338924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.338904    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"]
Sep 10 13:22:02.367174 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.367154    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k56m5\" (UniqueName: \"kubernetes.io/projected/a59d01b6-6020-4c97-aba9-d97700e632d4-kube-api-access-k56m5\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.367266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.367186    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a59d01b6-6020-4c97-aba9-d97700e632d4-tmp\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.367266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.367213    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a59d01b6-6020-4c97-aba9-d97700e632d4-manifests\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.367337 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.367272    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a59d01b6-6020-4c97-aba9-d97700e632d4-webhook-cert\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.367337 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.367300    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a59d01b6-6020-4c97-aba9-d97700e632d4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.467625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.467600    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a59d01b6-6020-4c97-aba9-d97700e632d4-manifests\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.467756 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.467639    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a59d01b6-6020-4c97-aba9-d97700e632d4-webhook-cert\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.467756 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.467692    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a59d01b6-6020-4c97-aba9-d97700e632d4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.467938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.467756    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k56m5\" (UniqueName: \"kubernetes.io/projected/a59d01b6-6020-4c97-aba9-d97700e632d4-kube-api-access-k56m5\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.467938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.467796    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a59d01b6-6020-4c97-aba9-d97700e632d4-tmp\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.468056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.467995    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a59d01b6-6020-4c97-aba9-d97700e632d4-manifests\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.469888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.469866    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a59d01b6-6020-4c97-aba9-d97700e632d4-tmp\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.470137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.470121    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a59d01b6-6020-4c97-aba9-d97700e632d4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.470201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.470181    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a59d01b6-6020-4c97-aba9-d97700e632d4-webhook-cert\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.484038 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.484013    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k56m5\" (UniqueName: \"kubernetes.io/projected/a59d01b6-6020-4c97-aba9-d97700e632d4-kube-api-access-k56m5\") pod \"opendatahub-operator-controller-manager-699db476d-p6hwj\" (UID: \"a59d01b6-6020-4c97-aba9-d97700e632d4\") " pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.594998 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.594935    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:02.889562 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.889488    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"]
Sep 10 13:22:02.892911 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:22:02.892883    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59d01b6_6020_4c97_aba9_d97700e632d4.slice/crio-67bb0ef2ccfe33744e71ca6706afaf4b79ffe5c60e73e69dcd4f012b4ca8c2ba WatchSource:0}: Error finding container 67bb0ef2ccfe33744e71ca6706afaf4b79ffe5c60e73e69dcd4f012b4ca8c2ba: Status 404 returned error can't find the container with id 67bb0ef2ccfe33744e71ca6706afaf4b79ffe5c60e73e69dcd4f012b4ca8c2ba
Sep 10 13:22:02.960423 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:02.960395    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:03.081296 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:03.081270    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:03.202451 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:03.202387    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:03.352960 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:03.352932    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:03.368847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:03.368823    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj" event={"ID":"a59d01b6-6020-4c97-aba9-d97700e632d4","Type":"ContainerStarted","Data":"67bb0ef2ccfe33744e71ca6706afaf4b79ffe5c60e73e69dcd4f012b4ca8c2ba"}
Sep 10 13:22:03.482328 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:03.482269    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:06.386253 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:06.386210    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj" event={"ID":"a59d01b6-6020-4c97-aba9-d97700e632d4","Type":"ContainerStarted","Data":"7f4bc1f4c6e8f63ac3fd796fe070183ea54fa8679a446b17132808741d3e7ac9"}
Sep 10 13:22:07.391413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:07.391379    2551 generic.go:356] "Generic (PLEG): container finished" podID="a59d01b6-6020-4c97-aba9-d97700e632d4" containerID="7f4bc1f4c6e8f63ac3fd796fe070183ea54fa8679a446b17132808741d3e7ac9" exitCode=0
Sep 10 13:22:07.391413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:07.391413    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj" event={"ID":"a59d01b6-6020-4c97-aba9-d97700e632d4","Type":"ContainerDied","Data":"7f4bc1f4c6e8f63ac3fd796fe070183ea54fa8679a446b17132808741d3e7ac9"}
Sep 10 13:22:08.397044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:08.397012    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj" event={"ID":"a59d01b6-6020-4c97-aba9-d97700e632d4","Type":"ContainerStarted","Data":"706d85c289b1c71230e198b1c89ee8a2b6d3e7c4215cb4c5e8dac55254ef120c"}
Sep 10 13:22:08.397394 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:08.397086    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:08.470007 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:08.469963    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj" podStartSLOduration=2.070689303 podStartE2EDuration="6.46994776s" totalImagesPullingTime="3.356388324s" totalInitContainerRuntime="1.042870133s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:08.465239375 +0000 UTC m=+293.676432654" watchObservedRunningTime="2026-09-10 13:22:08.46994776 +0000 UTC m=+293.681141016"
Sep 10 13:22:11.363280 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:11.363247    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78dbbc87c8-tgcj2"
Sep 10 13:22:15.128085 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:15.128052    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:22:15.132473 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:15.132448    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:22:15.136214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:15.136195    2551 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 13:22:15.883404 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:15.883367    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:15.964962 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:15.964936    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:17.552720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:17.552636    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:17.663193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:17.663158    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:17.770611 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:17.770586    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:19.402760 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:19.402731    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-699db476d-p6hwj"
Sep 10 13:22:23.255083 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:23.255040    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:23.450318 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:23.450287    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:23.620583 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:23.620558    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:23.957433 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:23.957361    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:24.052075 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.052047    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"]
Sep 10 13:22:24.055177 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.055161    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.059413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.059389    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dqclh\""
Sep 10 13:22:24.059413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.059405    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 13:22:24.059642 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.059390    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 13:22:24.063378 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.063360    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 13:22:24.063475 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.063380    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 13:22:24.080276 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.080255    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"]
Sep 10 13:22:24.115360 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.115334    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-485hk\" (UniqueName: \"kubernetes.io/projected/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-kube-api-access-485hk\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.115465 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.115378    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-tls-certs\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.115465 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.115396    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-tmp\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.136513 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.136486    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:24.216721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.216653    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-tls-certs\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.216721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.216693    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-tmp\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.216863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.216740    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-485hk\" (UniqueName: \"kubernetes.io/projected/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-kube-api-access-485hk\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.218870 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.218842    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-tmp\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.219075 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.219058    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-tls-certs\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.249233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.249207    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-485hk\" (UniqueName: \"kubernetes.io/projected/4fd220db-a802-4fe8-aaa5-2e9da1b476bf-kube-api-access-485hk\") pod \"kube-auth-proxy-69d5c84f88-m94bf\" (UID: \"4fd220db-a802-4fe8-aaa5-2e9da1b476bf\") " pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.364611 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.364581    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"
Sep 10 13:22:24.659383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.659353    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf"]
Sep 10 13:22:24.664314 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:22:24.664291    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd220db_a802_4fe8_aaa5_2e9da1b476bf.slice/crio-8fef1633b6efa8b2ac95d506e08f54cfd2c0fe600c5b7dd1985cfa27a66afffa WatchSource:0}: Error finding container 8fef1633b6efa8b2ac95d506e08f54cfd2c0fe600c5b7dd1985cfa27a66afffa: Status 404 returned error can't find the container with id 8fef1633b6efa8b2ac95d506e08f54cfd2c0fe600c5b7dd1985cfa27a66afffa
Sep 10 13:22:24.666062 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:24.666047    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:22:25.455988 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:25.455956    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf" event={"ID":"4fd220db-a802-4fe8-aaa5-2e9da1b476bf","Type":"ContainerStarted","Data":"8fef1633b6efa8b2ac95d506e08f54cfd2c0fe600c5b7dd1985cfa27a66afffa"}
Sep 10 13:22:28.470183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:28.470143    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf" event={"ID":"4fd220db-a802-4fe8-aaa5-2e9da1b476bf","Type":"ContainerStarted","Data":"49cc4d08cf80c7b348025430a2810bd0c7355e68858f84eab59a4d5766835cbe"}
Sep 10 13:22:28.513077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:28.513020    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69d5c84f88-m94bf" podStartSLOduration=1.435481316 podStartE2EDuration="4.513005365s" totalImagesPullingTime="3.077524049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:28.509768145 +0000 UTC m=+313.720961401" watchObservedRunningTime="2026-09-10 13:22:28.513005365 +0000 UTC m=+313.724198622"
Sep 10 13:22:28.551120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:28.551097    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:28.694454 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:28.694425    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:31.363226 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:31.363194    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:32.946570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:32.946536    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:33.054363 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:33.054335    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:33.485700 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:33.485652    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:33.606746 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:33.606702    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:33.732495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:33.732467    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:34.326059 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:34.326029    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:34.461197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:34.461170    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:34.693519 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:34.693402    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:34.799352 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:34.799328    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:35.286751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:35.286725    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:35.585067 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:35.584992    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:46.725816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.725778    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:46.793887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.793856    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"]
Sep 10 13:22:46.797195 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.797178    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:46.800593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.800575    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 13:22:46.800705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.800599    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-kpg42\""
Sep 10 13:22:46.821839 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.821819    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"]
Sep 10 13:22:46.865964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.865938    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:46.893044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.893022    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czcgj\" (UniqueName: \"kubernetes.io/projected/f7c9e53d-7df2-4098-acf7-3b1340a4cba4-kube-api-access-czcgj\") pod \"llmisvc-controller-manager-777c6bd7d5-fsmzw\" (UID: \"f7c9e53d-7df2-4098-acf7-3b1340a4cba4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:46.893127 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.893049    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7c9e53d-7df2-4098-acf7-3b1340a4cba4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fsmzw\" (UID: \"f7c9e53d-7df2-4098-acf7-3b1340a4cba4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:46.994387 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.994337    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czcgj\" (UniqueName: \"kubernetes.io/projected/f7c9e53d-7df2-4098-acf7-3b1340a4cba4-kube-api-access-czcgj\") pod \"llmisvc-controller-manager-777c6bd7d5-fsmzw\" (UID: \"f7c9e53d-7df2-4098-acf7-3b1340a4cba4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:46.994387 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.994365    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7c9e53d-7df2-4098-acf7-3b1340a4cba4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fsmzw\" (UID: \"f7c9e53d-7df2-4098-acf7-3b1340a4cba4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:46.996655 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:46.996633    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f7c9e53d-7df2-4098-acf7-3b1340a4cba4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-fsmzw\" (UID: \"f7c9e53d-7df2-4098-acf7-3b1340a4cba4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:47.013865 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:47.013845    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czcgj\" (UniqueName: \"kubernetes.io/projected/f7c9e53d-7df2-4098-acf7-3b1340a4cba4-kube-api-access-czcgj\") pod \"llmisvc-controller-manager-777c6bd7d5-fsmzw\" (UID: \"f7c9e53d-7df2-4098-acf7-3b1340a4cba4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:47.018610 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:47.018582    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:47.107249 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:47.107225    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:47.222564 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:47.222532    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:47.450981 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:47.450951    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:22:47.454866 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:22:47.454839    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7c9e53d_7df2_4098_acf7_3b1340a4cba4.slice/crio-70a77653e41ef78a786f5ea583a4e7611341002f89f26017687dc7cea87277b1 WatchSource:0}: Error finding container 70a77653e41ef78a786f5ea583a4e7611341002f89f26017687dc7cea87277b1: Status 404 returned error can't find the container with id 70a77653e41ef78a786f5ea583a4e7611341002f89f26017687dc7cea87277b1
Sep 10 13:22:47.467984 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:47.467962    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"]
Sep 10 13:22:47.538255 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:47.538228    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw" event={"ID":"f7c9e53d-7df2-4098-acf7-3b1340a4cba4","Type":"ContainerStarted","Data":"70a77653e41ef78a786f5ea583a4e7611341002f89f26017687dc7cea87277b1"}
Sep 10 13:22:50.553561 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:50.553534    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw" event={"ID":"f7c9e53d-7df2-4098-acf7-3b1340a4cba4","Type":"ContainerStarted","Data":"bcaae095c4c501dfd55cd7771181cb29006bfc79bc27b56f6b8f356b749e0e2b"}
Sep 10 13:22:50.553859 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:50.553630    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:22:50.641436 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:50.641357    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw" podStartSLOduration=1.7412731670000001 podStartE2EDuration="4.641338071s" totalImagesPullingTime="2.900064904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:22:50.640053525 +0000 UTC m=+335.851246782" watchObservedRunningTime="2026-09-10 13:22:50.641338071 +0000 UTC m=+335.852531327"
Sep 10 13:22:51.475833 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:22:51.475801    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:16.480996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:16.480920    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:16.566495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:16.566462    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:21.561131 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:21.561099    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-fsmzw"
Sep 10 13:23:35.319720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:35.319689    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:35.416879 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:35.416858    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:54.190878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:54.190844    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:57.049299 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:57.049265    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:57.471915 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:57.471836    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:23:57.813542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:23:57.813503    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:00.911183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:00.911146    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:01.476679 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:01.476632    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:01.771890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:01.771860    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:01.872049 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:01.872020    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:03.326955 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:03.326927    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:03.436909 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:03.436884    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:04.469899 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:04.469861    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:04.747526 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:04.747454    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:07.688048 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:07.688010    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:07.876036 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:07.876007    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:09.266423 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.266392    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:09.349792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.349758    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"]
Sep 10 13:24:09.352934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.352915    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.357614 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.357597    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 13:24:09.357870 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.357843    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lpc9q\""
Sep 10 13:24:09.359227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.359048    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 13:24:09.359227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.359120    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 13:24:09.359358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.359312    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 13:24:09.394156 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.394136    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"]
Sep 10 13:24:09.471424 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.471403    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5496b84d-b738-4751-add2-fc9eafbb4040-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.471528 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.471431    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhm7g\" (UniqueName: \"kubernetes.io/projected/5496b84d-b738-4751-add2-fc9eafbb4040-kube-api-access-zhm7g\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.471528 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.471513    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5496b84d-b738-4751-add2-fc9eafbb4040-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.572442 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.572418    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5496b84d-b738-4751-add2-fc9eafbb4040-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.572531 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.572458    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5496b84d-b738-4751-add2-fc9eafbb4040-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.572531 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.572480    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhm7g\" (UniqueName: \"kubernetes.io/projected/5496b84d-b738-4751-add2-fc9eafbb4040-kube-api-access-zhm7g\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.573069 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.573047    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5496b84d-b738-4751-add2-fc9eafbb4040-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.574786 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.574766    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5496b84d-b738-4751-add2-fc9eafbb4040-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.589031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.589007    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhm7g\" (UniqueName: \"kubernetes.io/projected/5496b84d-b738-4751-add2-fc9eafbb4040-kube-api-access-zhm7g\") pod \"kuadrant-console-plugin-fbdd4f445-plmr6\" (UID: \"5496b84d-b738-4751-add2-fc9eafbb4040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.633949 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.633927    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:09.661884 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.661864    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"
Sep 10 13:24:09.960053 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:09.959976    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6"]
Sep 10 13:24:09.963329 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:24:09.963303    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5496b84d_b738_4751_add2_fc9eafbb4040.slice/crio-aed55bf041de975fbd8dcbb18858e451b52c2f6cf88304212b68e8e916fc7c3d WatchSource:0}: Error finding container aed55bf041de975fbd8dcbb18858e451b52c2f6cf88304212b68e8e916fc7c3d: Status 404 returned error can't find the container with id aed55bf041de975fbd8dcbb18858e451b52c2f6cf88304212b68e8e916fc7c3d
Sep 10 13:24:10.868437 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:10.868404    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6" event={"ID":"5496b84d-b738-4751-add2-fc9eafbb4040","Type":"ContainerStarted","Data":"aed55bf041de975fbd8dcbb18858e451b52c2f6cf88304212b68e8e916fc7c3d"}
Sep 10 13:24:11.559691 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:11.559641    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:11.681575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:11.681540    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:11.785539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:11.785503    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:11.894994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:11.894916    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:14.887652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:14.887607    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6" event={"ID":"5496b84d-b738-4751-add2-fc9eafbb4040","Type":"ContainerStarted","Data":"408e5289c7572601ac7ffd948d051671a93b19b912afa02ebf5c7277f3aaf7da"}
Sep 10 13:24:14.926583 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:14.926538    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-plmr6" podStartSLOduration=1.169462026 podStartE2EDuration="5.926524128s" totalImagesPullingTime="4.757062102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:24:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:24:14.925417511 +0000 UTC m=+420.136610764" watchObservedRunningTime="2026-09-10 13:24:14.926524128 +0000 UTC m=+420.137717385"
Sep 10 13:24:19.963605 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:19.963569    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:20.060964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:20.060937    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:22.449210 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:22.449145    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:22.616082 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:22.616051    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:26.852705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:26.852654    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:26.973183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:26.973151    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:30.530711 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:30.530654    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:30.638677 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:30.638635    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:30.752125 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:30.752091    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:31.334953 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:31.334920    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:31.737959 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:31.737886    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:31.852179 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:31.852152    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:39.354650 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:39.354572    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:39.464352 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:39.464325    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:41.795267 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:41.795230    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:41.937331 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:41.937305    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:50.446158 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:50.446128    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:50.595728 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:50.595689    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:59.517155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:59.517123    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:24:59.618962 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:24:59.618934    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:03.597120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:03.597087    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:03.705204 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:03.705169    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:03.806109 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:03.806079    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:04.100810 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.100786    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:25:04.104048 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.104026    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.106790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.106769    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 13:25:04.114936 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.114915    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:25:04.138947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.138922    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:25:04.156256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.156234    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t47r8\" (UniqueName: \"kubernetes.io/projected/b342d82d-efe0-449b-87b8-020f7dad5bbe-kube-api-access-t47r8\") pod \"limitador-limitador-84994f7ffc-qlmdf\" (UID: \"b342d82d-efe0-449b-87b8-020f7dad5bbe\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.156335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.156266    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b342d82d-efe0-449b-87b8-020f7dad5bbe-config-file\") pod \"limitador-limitador-84994f7ffc-qlmdf\" (UID: \"b342d82d-efe0-449b-87b8-020f7dad5bbe\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.257562 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.257538    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b342d82d-efe0-449b-87b8-020f7dad5bbe-config-file\") pod \"limitador-limitador-84994f7ffc-qlmdf\" (UID: \"b342d82d-efe0-449b-87b8-020f7dad5bbe\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.257642 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.257615    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t47r8\" (UniqueName: \"kubernetes.io/projected/b342d82d-efe0-449b-87b8-020f7dad5bbe-kube-api-access-t47r8\") pod \"limitador-limitador-84994f7ffc-qlmdf\" (UID: \"b342d82d-efe0-449b-87b8-020f7dad5bbe\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.258119 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.258099    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b342d82d-efe0-449b-87b8-020f7dad5bbe-config-file\") pod \"limitador-limitador-84994f7ffc-qlmdf\" (UID: \"b342d82d-efe0-449b-87b8-020f7dad5bbe\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.266643 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.266615    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t47r8\" (UniqueName: \"kubernetes.io/projected/b342d82d-efe0-449b-87b8-020f7dad5bbe-kube-api-access-t47r8\") pod \"limitador-limitador-84994f7ffc-qlmdf\" (UID: \"b342d82d-efe0-449b-87b8-020f7dad5bbe\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.413087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.413027    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:04.477427 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.477223    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:04.565178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.565155    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx65w"]
Sep 10 13:25:04.568429 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.568413    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx65w"
Sep 10 13:25:04.571376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.571360    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-t652d\""
Sep 10 13:25:04.576209 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.576188    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx65w"]
Sep 10 13:25:04.640159 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.640138    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:04.661199 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.661178    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7d7m\" (UniqueName: \"kubernetes.io/projected/e238ed17-1b4c-4df1-b208-118d6e2e9b82-kube-api-access-b7d7m\") pod \"authorino-59c974bd48-cx65w\" (UID: \"e238ed17-1b4c-4df1-b208-118d6e2e9b82\") " pod="kuadrant-system/authorino-59c974bd48-cx65w"
Sep 10 13:25:04.684902 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.684850    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:25:04.688987 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:25:04.688968    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb342d82d_efe0_449b_87b8_020f7dad5bbe.slice/crio-56c0abc06adb9a84b556f0d37ced29ae1c720d9d18368543dda4ecdd3a5d9784 WatchSource:0}: Error finding container 56c0abc06adb9a84b556f0d37ced29ae1c720d9d18368543dda4ecdd3a5d9784: Status 404 returned error can't find the container with id 56c0abc06adb9a84b556f0d37ced29ae1c720d9d18368543dda4ecdd3a5d9784
Sep 10 13:25:04.761761 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.761737    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7d7m\" (UniqueName: \"kubernetes.io/projected/e238ed17-1b4c-4df1-b208-118d6e2e9b82-kube-api-access-b7d7m\") pod \"authorino-59c974bd48-cx65w\" (UID: \"e238ed17-1b4c-4df1-b208-118d6e2e9b82\") " pod="kuadrant-system/authorino-59c974bd48-cx65w"
Sep 10 13:25:04.770614 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.770593    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7d7m\" (UniqueName: \"kubernetes.io/projected/e238ed17-1b4c-4df1-b208-118d6e2e9b82-kube-api-access-b7d7m\") pod \"authorino-59c974bd48-cx65w\" (UID: \"e238ed17-1b4c-4df1-b208-118d6e2e9b82\") " pod="kuadrant-system/authorino-59c974bd48-cx65w"
Sep 10 13:25:04.877758 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:04.877738    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx65w"
Sep 10 13:25:05.086888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:05.086856    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf" event={"ID":"b342d82d-efe0-449b-87b8-020f7dad5bbe","Type":"ContainerStarted","Data":"56c0abc06adb9a84b556f0d37ced29ae1c720d9d18368543dda4ecdd3a5d9784"}
Sep 10 13:25:05.146576 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:05.146554    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx65w"]
Sep 10 13:25:05.150254 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:25:05.150234    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode238ed17_1b4c_4df1_b208_118d6e2e9b82.slice/crio-0c4520dd0cec36cb102f7b48725fcaa2f38d23f9d94872ff9847c12fb76395cf WatchSource:0}: Error finding container 0c4520dd0cec36cb102f7b48725fcaa2f38d23f9d94872ff9847c12fb76395cf: Status 404 returned error can't find the container with id 0c4520dd0cec36cb102f7b48725fcaa2f38d23f9d94872ff9847c12fb76395cf
Sep 10 13:25:06.094255 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:06.094208    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx65w" event={"ID":"e238ed17-1b4c-4df1-b208-118d6e2e9b82","Type":"ContainerStarted","Data":"0c4520dd0cec36cb102f7b48725fcaa2f38d23f9d94872ff9847c12fb76395cf"}
Sep 10 13:25:08.106916 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:08.106874    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf" event={"ID":"b342d82d-efe0-449b-87b8-020f7dad5bbe","Type":"ContainerStarted","Data":"ed36641a97c247c694f97e58db4b0ee7005a3d30780e1f28095ef0d768d11439"}
Sep 10 13:25:08.107396 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:08.107029    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:08.128187 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:08.128139    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf" podStartSLOduration=1.7701651360000001 podStartE2EDuration="4.128124851s" totalImagesPullingTime="2.357959715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:25:08.127893201 +0000 UTC m=+473.339086461" watchObservedRunningTime="2026-09-10 13:25:08.128124851 +0000 UTC m=+473.339318124"
Sep 10 13:25:09.113001 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:09.112965    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx65w" event={"ID":"e238ed17-1b4c-4df1-b208-118d6e2e9b82","Type":"ContainerStarted","Data":"3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061"}
Sep 10 13:25:09.133040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:09.132992    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-cx65w" podStartSLOduration=2.085203087 podStartE2EDuration="5.132972238s" totalImagesPullingTime="3.047769151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:25:09.131735751 +0000 UTC m=+474.342929009" watchObservedRunningTime="2026-09-10 13:25:09.132972238 +0000 UTC m=+474.344165498"
Sep 10 13:25:19.114420 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:19.114389    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"
Sep 10 13:25:24.130871 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:24.130838    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:24.231924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:24.231896    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:24.649096 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:24.649038    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:26.963316 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:26.963276    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:27.064250 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:27.064223    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:37.656727 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:37.656694    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:38.095956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.095893    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:38.230990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.230963    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-764b4d4bfd-d6glk"]
Sep 10 13:25:38.234410 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.234392    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.237595 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.237578    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 13:25:38.244474 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.244453    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-764b4d4bfd-d6glk"]
Sep 10 13:25:38.308949 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.308926    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dce3576c-fdb9-49f0-b082-bea86933f43f-tls-cert\") pod \"authorino-764b4d4bfd-d6glk\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") " pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.309056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.308963    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zttm\" (UniqueName: \"kubernetes.io/projected/dce3576c-fdb9-49f0-b082-bea86933f43f-kube-api-access-4zttm\") pod \"authorino-764b4d4bfd-d6glk\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") " pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.410167 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.410112    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zttm\" (UniqueName: \"kubernetes.io/projected/dce3576c-fdb9-49f0-b082-bea86933f43f-kube-api-access-4zttm\") pod \"authorino-764b4d4bfd-d6glk\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") " pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.410259 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.410187    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dce3576c-fdb9-49f0-b082-bea86933f43f-tls-cert\") pod \"authorino-764b4d4bfd-d6glk\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") " pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.412457 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.412439    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dce3576c-fdb9-49f0-b082-bea86933f43f-tls-cert\") pod \"authorino-764b4d4bfd-d6glk\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") " pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.420839 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.420815    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zttm\" (UniqueName: \"kubernetes.io/projected/dce3576c-fdb9-49f0-b082-bea86933f43f-kube-api-access-4zttm\") pod \"authorino-764b4d4bfd-d6glk\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") " pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.543699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.543678    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:25:38.816967 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:38.816943    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-764b4d4bfd-d6glk"]
Sep 10 13:25:38.818852 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:25:38.818822    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddce3576c_fdb9_49f0_b082_bea86933f43f.slice/crio-83d80cf68be8a89ea3d62fb3e9cc26bf6a5dba28089db457fb095619d44e386d WatchSource:0}: Error finding container 83d80cf68be8a89ea3d62fb3e9cc26bf6a5dba28089db457fb095619d44e386d: Status 404 returned error can't find the container with id 83d80cf68be8a89ea3d62fb3e9cc26bf6a5dba28089db457fb095619d44e386d
Sep 10 13:25:39.239489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:39.239410    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-764b4d4bfd-d6glk" event={"ID":"dce3576c-fdb9-49f0-b082-bea86933f43f","Type":"ContainerStarted","Data":"83d80cf68be8a89ea3d62fb3e9cc26bf6a5dba28089db457fb095619d44e386d"}
Sep 10 13:25:40.244940 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.244905    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-764b4d4bfd-d6glk" event={"ID":"dce3576c-fdb9-49f0-b082-bea86933f43f","Type":"ContainerStarted","Data":"dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752"}
Sep 10 13:25:40.308674 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.308610    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-764b4d4bfd-d6glk" podStartSLOduration=1.7664807 podStartE2EDuration="2.308588062s" totalImagesPullingTime="542.107362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:25:40.265554428 +0000 UTC m=+505.476747685" watchObservedRunningTime="2026-09-10 13:25:40.308588062 +0000 UTC m=+505.519781318"
Sep 10 13:25:40.308850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.308833    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx65w"]
Sep 10 13:25:40.309014 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.308995    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-cx65w" podUID="e238ed17-1b4c-4df1-b208-118d6e2e9b82" containerName="authorino" containerID="cri-o://3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061" gracePeriod=30
Sep 10 13:25:40.545009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.544988    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx65w"
Sep 10 13:25:40.626821 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.626794    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7d7m\" (UniqueName: \"kubernetes.io/projected/e238ed17-1b4c-4df1-b208-118d6e2e9b82-kube-api-access-b7d7m\") pod \"e238ed17-1b4c-4df1-b208-118d6e2e9b82\" (UID: \"e238ed17-1b4c-4df1-b208-118d6e2e9b82\") "
Sep 10 13:25:40.628853 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.628826    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e238ed17-1b4c-4df1-b208-118d6e2e9b82-kube-api-access-b7d7m" (OuterVolumeSpecName: "kube-api-access-b7d7m") pod "e238ed17-1b4c-4df1-b208-118d6e2e9b82" (UID: "e238ed17-1b4c-4df1-b208-118d6e2e9b82"). InnerVolumeSpecName "kube-api-access-b7d7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:25:40.727857 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:40.727834    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7d7m\" (UniqueName: \"kubernetes.io/projected/e238ed17-1b4c-4df1-b208-118d6e2e9b82-kube-api-access-b7d7m\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:25:41.249737 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.249708    2551 generic.go:356] "Generic (PLEG): container finished" podID="e238ed17-1b4c-4df1-b208-118d6e2e9b82" containerID="3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061" exitCode=0
Sep 10 13:25:41.250157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.249740    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-cx65w"
Sep 10 13:25:41.250157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.249782    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx65w" event={"ID":"e238ed17-1b4c-4df1-b208-118d6e2e9b82","Type":"ContainerDied","Data":"3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061"}
Sep 10 13:25:41.250157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.249827    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-cx65w" event={"ID":"e238ed17-1b4c-4df1-b208-118d6e2e9b82","Type":"ContainerDied","Data":"0c4520dd0cec36cb102f7b48725fcaa2f38d23f9d94872ff9847c12fb76395cf"}
Sep 10 13:25:41.250157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.249860    2551 scope.go:117] "RemoveContainer" containerID="3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061"
Sep 10 13:25:41.262648 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.262617    2551 scope.go:117] "RemoveContainer" containerID="3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061"
Sep 10 13:25:41.263225 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:25:41.263201    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061\": container with ID starting with 3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061 not found: ID does not exist" containerID="3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061"
Sep 10 13:25:41.263317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.263236    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061"} err="failed to get container status \"3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061\": rpc error: code = NotFound desc = could not find container \"3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061\": container with ID starting with 3a825fc5134519fb204702376149e4228b5b473c8db3ece3ab4ed3456ec40061 not found: ID does not exist"
Sep 10 13:25:41.279742 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.279721    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx65w"]
Sep 10 13:25:41.284227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:41.284211    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-cx65w"]
Sep 10 13:25:43.247241 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:43.247205    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e238ed17-1b4c-4df1-b208-118d6e2e9b82" path="/var/lib/kubelet/pods/e238ed17-1b4c-4df1-b208-118d6e2e9b82/volumes"
Sep 10 13:25:44.491159 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:44.491129    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:45.547905 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:45.547871    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:45.646741 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:45.646712    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:45.901918 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:45.901836    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:45.971508 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:45.971478    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:48.963845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:48.963808    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:25:49.064268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:25:49.064236    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:09.833408 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:09.833310    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:09.909557 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:09.909535    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:20.584053 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:20.584019    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:20.684536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:20.684498    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:22.009701 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:22.009656    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:22.110125 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:22.110096    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:22.210536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:22.210455    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:22.469615 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:22.469530    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:22.665993 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:22.665961    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:22.759703 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:22.759610    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:22.916538 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:22.916502    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:26:33.619969 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:33.619938    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:33.742518 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:33.742493    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:38.964490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:38.964453    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:39.702612 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:39.702575    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:26:43.350112 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:43.350078    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:44.092762 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:44.092724    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:26:50.584655 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:50.584621    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:51.691340 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:51.691299    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:26:56.947034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:56.947004    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:26:58.183709 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:26:58.183654    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:27:06.463462 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:06.463432    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:27:08.091513 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:08.091468    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:27:10.279304 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:10.279236    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:27:15.015514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:15.015483    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:27:15.154433 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:15.154404    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:27:15.157519 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:15.157499    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:27:24.932021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.931986    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"]
Sep 10 13:27:24.932524 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.932285    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e238ed17-1b4c-4df1-b208-118d6e2e9b82" containerName="authorino"
Sep 10 13:27:24.932524 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.932296    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e238ed17-1b4c-4df1-b208-118d6e2e9b82" containerName="authorino"
Sep 10 13:27:24.932524 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.932346    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e238ed17-1b4c-4df1-b208-118d6e2e9b82" containerName="authorino"
Sep 10 13:27:24.935275 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.935255    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:24.939483 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.939458    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g6wbm\""
Sep 10 13:27:24.939620 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.939480    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 13:27:24.939775 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.939757    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 13:27:24.939914 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.939888    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 13:27:24.953571 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.953551    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"]
Sep 10 13:27:24.981888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:24.981864    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:27:25.094740 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.094719    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.094845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.094745    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.094845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.094777    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brpmd\" (UniqueName: \"kubernetes.io/projected/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-kube-api-access-brpmd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.094845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.094814    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.094990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.094848    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.094990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.094932    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.195553 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.195477    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.195553 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.195545    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.195773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.195586    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.195773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.195609    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.195773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.195648    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brpmd\" (UniqueName: \"kubernetes.io/projected/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-kube-api-access-brpmd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.195773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.195691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.195993 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.195972    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.196059 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.196003    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.196113 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.196072    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.197845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.197827    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.198033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.198015    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.204990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.204971    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brpmd\" (UniqueName: \"kubernetes.io/projected/ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60-kube-api-access-brpmd\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-jg5rb\" (UID: \"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.246120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.246089    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:25.517867 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.517842    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"]
Sep 10 13:27:25.520023 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:27:25.519989    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba75e13a_05ee_4b30_ae1f_d50f5dfcaf60.slice/crio-e3c641c019e48cd101f9d9a2d6fca1ead36e0c0f3fbaf201827832ffd2cb9e92 WatchSource:0}: Error finding container e3c641c019e48cd101f9d9a2d6fca1ead36e0c0f3fbaf201827832ffd2cb9e92: Status 404 returned error can't find the container with id e3c641c019e48cd101f9d9a2d6fca1ead36e0c0f3fbaf201827832ffd2cb9e92
Sep 10 13:27:25.521515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.521488    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:27:25.682993 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:25.682961    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb" event={"ID":"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60","Type":"ContainerStarted","Data":"e3c641c019e48cd101f9d9a2d6fca1ead36e0c0f3fbaf201827832ffd2cb9e92"}
Sep 10 13:27:26.599374 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:26.599350    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:27:26.689547 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:26.689506    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb" event={"ID":"ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60","Type":"ContainerStarted","Data":"be754b89185d18e50afbe94129920e1ecc3fea03c45d710a71efb97c6ddb2260"}
Sep 10 13:27:26.715601 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:26.715552    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb" podStartSLOduration=1.701462628 podStartE2EDuration="2.715535694s" totalImagesPullingTime="1.014073066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:26.712965455 +0000 UTC m=+611.924158711" watchObservedRunningTime="2026-09-10 13:27:26.715535694 +0000 UTC m=+611.926728951"
Sep 10 13:27:35.252477 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:35.252444    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:35.252866 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:35.252493    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:35.261305 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:35.261281    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:35.735785 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:35.735761    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-jg5rb"
Sep 10 13:27:41.727966 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.727887    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"]
Sep 10 13:27:41.731833 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.731815    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.734832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.734810    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 13:27:41.746351 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.746332    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"]
Sep 10 13:27:41.778876 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.778843    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:27:41.812934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.812906    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.813034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.812939    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f99v5\" (UniqueName: \"kubernetes.io/projected/be1370a7-30c7-4864-aecb-799fd45f3f8d-kube-api-access-f99v5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.813034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.812965    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be1370a7-30c7-4864-aecb-799fd45f3f8d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.813034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.812986    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.813155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.813060    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.813155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.813096    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.913629 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.913600    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be1370a7-30c7-4864-aecb-799fd45f3f8d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.913751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.913641    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.913751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.913715    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.913869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.913756    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.913869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.913803    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.913869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.913827    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f99v5\" (UniqueName: \"kubernetes.io/projected/be1370a7-30c7-4864-aecb-799fd45f3f8d-kube-api-access-f99v5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.914170 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.914144    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.914268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.914230    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.914329 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.914269    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.915959 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.915932    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be1370a7-30c7-4864-aecb-799fd45f3f8d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.916027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.916012    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be1370a7-30c7-4864-aecb-799fd45f3f8d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:41.922950 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:41.922923    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f99v5\" (UniqueName: \"kubernetes.io/projected/be1370a7-30c7-4864-aecb-799fd45f3f8d-kube-api-access-f99v5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lb958\" (UID: \"be1370a7-30c7-4864-aecb-799fd45f3f8d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:42.040533 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:42.040510    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:42.324500 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:42.324477    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"]
Sep 10 13:27:42.326382 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:27:42.326356    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe1370a7_30c7_4864_aecb_799fd45f3f8d.slice/crio-a2c12664f9c75aa9984c988f41ade8337491ae08e61dc4087fd1fe565ee9a49e WatchSource:0}: Error finding container a2c12664f9c75aa9984c988f41ade8337491ae08e61dc4087fd1fe565ee9a49e: Status 404 returned error can't find the container with id a2c12664f9c75aa9984c988f41ade8337491ae08e61dc4087fd1fe565ee9a49e
Sep 10 13:27:42.755332 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:42.755300    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958" event={"ID":"be1370a7-30c7-4864-aecb-799fd45f3f8d","Type":"ContainerStarted","Data":"d8d89f7367f8b5ac37c22184312111ed5b3b5a8a656489e7510031366f8a586b"}
Sep 10 13:27:42.755754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:42.755345    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958" event={"ID":"be1370a7-30c7-4864-aecb-799fd45f3f8d","Type":"ContainerStarted","Data":"a2c12664f9c75aa9984c988f41ade8337491ae08e61dc4087fd1fe565ee9a49e"}
Sep 10 13:27:42.792854 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:42.792805    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958" podStartSLOduration=1.583127486 podStartE2EDuration="1.7927906s" totalImagesPullingTime="209.663114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:27:42.790944309 +0000 UTC m=+628.002137589" watchObservedRunningTime="2026-09-10 13:27:42.7927906 +0000 UTC m=+628.003983858"
Sep 10 13:27:44.094320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:44.094285    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:27:52.041334 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:52.041299    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:52.041334 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:52.041338    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:52.054324 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:52.054303    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:27:52.808143 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:27:52.808101    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lb958"
Sep 10 13:28:07.557439 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:07.557405    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:28:23.298749 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:23.298716    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:28:31.065957 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:31.065920    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:28:32.725105 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:32.725071    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:28:58.376992 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.376960    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f8d55b679-mnnjk"]
Sep 10 13:28:58.379936 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.379919    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.393197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.393174    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f8d55b679-mnnjk"]
Sep 10 13:28:58.479522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.479499    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt2zh\" (UniqueName: \"kubernetes.io/projected/f0b01acd-281c-45f7-b935-54a0ed20fa0e-kube-api-access-mt2zh\") pod \"authorino-f8d55b679-mnnjk\" (UID: \"f0b01acd-281c-45f7-b935-54a0ed20fa0e\") " pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.479640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.479548    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f0b01acd-281c-45f7-b935-54a0ed20fa0e-tls-cert\") pod \"authorino-f8d55b679-mnnjk\" (UID: \"f0b01acd-281c-45f7-b935-54a0ed20fa0e\") " pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.580237 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.580214    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f0b01acd-281c-45f7-b935-54a0ed20fa0e-tls-cert\") pod \"authorino-f8d55b679-mnnjk\" (UID: \"f0b01acd-281c-45f7-b935-54a0ed20fa0e\") " pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.580350 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.580268    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt2zh\" (UniqueName: \"kubernetes.io/projected/f0b01acd-281c-45f7-b935-54a0ed20fa0e-kube-api-access-mt2zh\") pod \"authorino-f8d55b679-mnnjk\" (UID: \"f0b01acd-281c-45f7-b935-54a0ed20fa0e\") " pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.582537 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.582517    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f0b01acd-281c-45f7-b935-54a0ed20fa0e-tls-cert\") pod \"authorino-f8d55b679-mnnjk\" (UID: \"f0b01acd-281c-45f7-b935-54a0ed20fa0e\") " pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.593198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.593175    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt2zh\" (UniqueName: \"kubernetes.io/projected/f0b01acd-281c-45f7-b935-54a0ed20fa0e-kube-api-access-mt2zh\") pod \"authorino-f8d55b679-mnnjk\" (UID: \"f0b01acd-281c-45f7-b935-54a0ed20fa0e\") " pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.688790 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.688740    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f8d55b679-mnnjk"
Sep 10 13:28:58.961487 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:58.961368    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f8d55b679-mnnjk"]
Sep 10 13:28:58.964180 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:28:58.964142    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0b01acd_281c_45f7_b935_54a0ed20fa0e.slice/crio-1a87194b0e7b79d7f29dfb32511fc77a07b0cea52e8c0c33f6871672c8d8cfee WatchSource:0}: Error finding container 1a87194b0e7b79d7f29dfb32511fc77a07b0cea52e8c0c33f6871672c8d8cfee: Status 404 returned error can't find the container with id 1a87194b0e7b79d7f29dfb32511fc77a07b0cea52e8c0c33f6871672c8d8cfee
Sep 10 13:28:59.083354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:28:59.083327    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f8d55b679-mnnjk" event={"ID":"f0b01acd-281c-45f7-b935-54a0ed20fa0e","Type":"ContainerStarted","Data":"1a87194b0e7b79d7f29dfb32511fc77a07b0cea52e8c0c33f6871672c8d8cfee"}
Sep 10 13:29:00.092622 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.091887    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f8d55b679-mnnjk" event={"ID":"f0b01acd-281c-45f7-b935-54a0ed20fa0e","Type":"ContainerStarted","Data":"040664ad58c604c482136fcce429113c6cdf81ff9b7b7a9fa77d167623072b3d"}
Sep 10 13:29:00.115223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.115162    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f8d55b679-mnnjk" podStartSLOduration=1.6356005260000002 podStartE2EDuration="2.115136961s" totalImagesPullingTime="479.536435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:00.112560481 +0000 UTC m=+705.323753739" watchObservedRunningTime="2026-09-10 13:29:00.115136961 +0000 UTC m=+705.326330220"
Sep 10 13:29:00.149547 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.149510    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-764b4d4bfd-d6glk"]
Sep 10 13:29:00.149971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.149940    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-764b4d4bfd-d6glk" podUID="dce3576c-fdb9-49f0-b082-bea86933f43f" containerName="authorino" containerID="cri-o://dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752" gracePeriod=30
Sep 10 13:29:00.396576 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.396549    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:29:00.497364 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.497318    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zttm\" (UniqueName: \"kubernetes.io/projected/dce3576c-fdb9-49f0-b082-bea86933f43f-kube-api-access-4zttm\") pod \"dce3576c-fdb9-49f0-b082-bea86933f43f\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") "
Sep 10 13:29:00.497831 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.497802    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dce3576c-fdb9-49f0-b082-bea86933f43f-tls-cert\") pod \"dce3576c-fdb9-49f0-b082-bea86933f43f\" (UID: \"dce3576c-fdb9-49f0-b082-bea86933f43f\") "
Sep 10 13:29:00.504681 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.504095    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dce3576c-fdb9-49f0-b082-bea86933f43f-kube-api-access-4zttm" (OuterVolumeSpecName: "kube-api-access-4zttm") pod "dce3576c-fdb9-49f0-b082-bea86933f43f" (UID: "dce3576c-fdb9-49f0-b082-bea86933f43f"). InnerVolumeSpecName "kube-api-access-4zttm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:29:00.513656 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.513628    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dce3576c-fdb9-49f0-b082-bea86933f43f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "dce3576c-fdb9-49f0-b082-bea86933f43f" (UID: "dce3576c-fdb9-49f0-b082-bea86933f43f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:29:00.599570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.599540    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zttm\" (UniqueName: \"kubernetes.io/projected/dce3576c-fdb9-49f0-b082-bea86933f43f-kube-api-access-4zttm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:29:00.599570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.599567    2551 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dce3576c-fdb9-49f0-b082-bea86933f43f-tls-cert\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:29:00.745539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.745461    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:00.845926 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:00.845894    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:01.098328 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.098293    2551 generic.go:356] "Generic (PLEG): container finished" podID="dce3576c-fdb9-49f0-b082-bea86933f43f" containerID="dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752" exitCode=0
Sep 10 13:29:01.098765 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.098355    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-764b4d4bfd-d6glk"
Sep 10 13:29:01.098765 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.098376    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-764b4d4bfd-d6glk" event={"ID":"dce3576c-fdb9-49f0-b082-bea86933f43f","Type":"ContainerDied","Data":"dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752"}
Sep 10 13:29:01.098765 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.098416    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-764b4d4bfd-d6glk" event={"ID":"dce3576c-fdb9-49f0-b082-bea86933f43f","Type":"ContainerDied","Data":"83d80cf68be8a89ea3d62fb3e9cc26bf6a5dba28089db457fb095619d44e386d"}
Sep 10 13:29:01.098765 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.098434    2551 scope.go:117] "RemoveContainer" containerID="dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752"
Sep 10 13:29:01.106719 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.106701    2551 scope.go:117] "RemoveContainer" containerID="dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752"
Sep 10 13:29:01.106974 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:01.106957    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752\": container with ID starting with dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752 not found: ID does not exist" containerID="dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752"
Sep 10 13:29:01.107030 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.106981    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752"} err="failed to get container status \"dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752\": rpc error: code = NotFound desc = could not find container \"dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752\": container with ID starting with dfd423fe6b0d0d744d50c9d8aa7d739d40b639178a4cbdb80a1ff1b483c81752 not found: ID does not exist"
Sep 10 13:29:01.125365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.125345    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-764b4d4bfd-d6glk"]
Sep 10 13:29:01.129566 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.129544    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-764b4d4bfd-d6glk"]
Sep 10 13:29:01.248146 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:01.248113    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dce3576c-fdb9-49f0-b082-bea86933f43f" path="/var/lib/kubelet/pods/dce3576c-fdb9-49f0-b082-bea86933f43f/volumes"
Sep 10 13:29:02.041403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:02.041371    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:02.142142 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:02.142107    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:21.981507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:21.981469    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:22.078023 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:22.077991    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:22.299869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:22.299831    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:22.402746 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:22.402711    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:23.565513 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:23.565471    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:23.663046 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:23.663013    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:24.136314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.136279    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:24.318177 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.318140    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7845-6b7f5a"]
Sep 10 13:29:24.318532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.318516    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dce3576c-fdb9-49f0-b082-bea86933f43f" containerName="authorino"
Sep 10 13:29:24.318624 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.318536    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="dce3576c-fdb9-49f0-b082-bea86933f43f" containerName="authorino"
Sep 10 13:29:24.318698 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.318644    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="dce3576c-fdb9-49f0-b082-bea86933f43f" containerName="authorino"
Sep 10 13:29:24.322694 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.322653    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:24.326211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.326194    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tqk99\""
Sep 10 13:29:24.331145 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.331122    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-6b7f5a"]
Sep 10 13:29:24.375037 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.375010    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r926v\" (UniqueName: \"kubernetes.io/projected/5123b281-8e62-4ce3-8a89-79990684e0a6-kube-api-access-r926v\") pod \"test-curl-7845-6b7f5a\" (UID: \"5123b281-8e62-4ce3-8a89-79990684e0a6\") " pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:24.475737 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.475681    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r926v\" (UniqueName: \"kubernetes.io/projected/5123b281-8e62-4ce3-8a89-79990684e0a6-kube-api-access-r926v\") pod \"test-curl-7845-6b7f5a\" (UID: \"5123b281-8e62-4ce3-8a89-79990684e0a6\") " pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:24.486671 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.486639    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r926v\" (UniqueName: \"kubernetes.io/projected/5123b281-8e62-4ce3-8a89-79990684e0a6-kube-api-access-r926v\") pod \"test-curl-7845-6b7f5a\" (UID: \"5123b281-8e62-4ce3-8a89-79990684e0a6\") " pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:24.631948 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.631921    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:24.904529 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:24.904506    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-6b7f5a"]
Sep 10 13:29:24.906733 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:24.906705    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5123b281_8e62_4ce3_8a89_79990684e0a6.slice/crio-731950de77f8545a43250fffbfb1096a5e76084f7cdd8bad9622863652a16392 WatchSource:0}: Error finding container 731950de77f8545a43250fffbfb1096a5e76084f7cdd8bad9622863652a16392: Status 404 returned error can't find the container with id 731950de77f8545a43250fffbfb1096a5e76084f7cdd8bad9622863652a16392
Sep 10 13:29:25.204497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:25.204421    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-6b7f5a" event={"ID":"5123b281-8e62-4ce3-8a89-79990684e0a6","Type":"ContainerStarted","Data":"731950de77f8545a43250fffbfb1096a5e76084f7cdd8bad9622863652a16392"}
Sep 10 13:29:25.720983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:25.720948    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:29:26.096354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:26.096310    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:26.211406 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:26.211371    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:27.215016 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:27.214988    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-6b7f5a" event={"ID":"5123b281-8e62-4ce3-8a89-79990684e0a6","Type":"ContainerStarted","Data":"e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb"}
Sep 10 13:29:27.298325 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:27.298260    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7845-6b7f5a" podStartSLOduration=1.217709837 podStartE2EDuration="3.298233099s" totalImagesPullingTime="2.080523262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:27.296783392 +0000 UTC m=+732.507976662" watchObservedRunningTime="2026-09-10 13:29:27.298233099 +0000 UTC m=+732.509426359"
Sep 10 13:29:27.596339 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:27.596259    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:27.686773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:27.686746    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:28.059412 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.059377    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7845-6b7f5a"]
Sep 10 13:29:28.062234 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.062201    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7845-6b7f5a"]
Sep 10 13:29:28.206329 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.206299    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7845-8d00f8"]
Sep 10 13:29:28.206601 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.206590    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5123b281-8e62-4ce3-8a89-79990684e0a6" containerName="test-curl-7845-6b7f5a"
Sep 10 13:29:28.206640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.206603    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5123b281-8e62-4ce3-8a89-79990684e0a6" containerName="test-curl-7845-6b7f5a"
Sep 10 13:29:28.206689 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.206684    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5123b281-8e62-4ce3-8a89-79990684e0a6" containerName="test-curl-7845-6b7f5a"
Sep 10 13:29:28.209701 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.209682    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:28.220929 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.220900    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-8d00f8"]
Sep 10 13:29:28.228679 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:28.228629    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7845-6b7f5a" container="test-curl-7845-6b7f5a"
Sep 10 13:29:28.250993 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.250970    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:28.307598 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.307577    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd5kv\" (UniqueName: \"kubernetes.io/projected/bebb00ee-8b8f-405e-acb3-586af8608193-kube-api-access-hd5kv\") pod \"test-curl-7845-8d00f8\" (UID: \"bebb00ee-8b8f-405e-acb3-586af8608193\") " pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:28.354423 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.354368    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:28.408601 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.408579    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd5kv\" (UniqueName: \"kubernetes.io/projected/bebb00ee-8b8f-405e-acb3-586af8608193-kube-api-access-hd5kv\") pod \"test-curl-7845-8d00f8\" (UID: \"bebb00ee-8b8f-405e-acb3-586af8608193\") " pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:28.423053 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.423030    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd5kv\" (UniqueName: \"kubernetes.io/projected/bebb00ee-8b8f-405e-acb3-586af8608193-kube-api-access-hd5kv\") pod \"test-curl-7845-8d00f8\" (UID: \"bebb00ee-8b8f-405e-acb3-586af8608193\") " pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:28.527559 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.527539    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:28.798610 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:28.798585    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7845-8d00f8"]
Sep 10 13:29:28.801221 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:28.801193    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbebb00ee_8b8f_405e_acb3_586af8608193.slice/crio-0330f83da45b1cca3a147bfdedcc6cb86e18ddc1fd76f2c209e5cf3d9599d503 WatchSource:0}: Error finding container 0330f83da45b1cca3a147bfdedcc6cb86e18ddc1fd76f2c209e5cf3d9599d503: Status 404 returned error can't find the container with id 0330f83da45b1cca3a147bfdedcc6cb86e18ddc1fd76f2c209e5cf3d9599d503
Sep 10 13:29:29.234638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:29.234547    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-8d00f8" event={"ID":"bebb00ee-8b8f-405e-acb3-586af8608193","Type":"ContainerStarted","Data":"ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9"}
Sep 10 13:29:29.234638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:29.234586    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7845-8d00f8" event={"ID":"bebb00ee-8b8f-405e-acb3-586af8608193","Type":"ContainerStarted","Data":"0330f83da45b1cca3a147bfdedcc6cb86e18ddc1fd76f2c209e5cf3d9599d503"}
Sep 10 13:29:29.235180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:29.234727    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7845-6b7f5a" podUID="5123b281-8e62-4ce3-8a89-79990684e0a6" containerName="test-curl-7845-6b7f5a" containerID="cri-o://e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb" gracePeriod=2
Sep 10 13:29:29.258302 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:29.258258    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7845-8d00f8" podStartSLOduration=1.258235824 podStartE2EDuration="1.258235824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:29.256065706 +0000 UTC m=+734.467258964" watchObservedRunningTime="2026-09-10 13:29:29.258235824 +0000 UTC m=+734.469429081"
Sep 10 13:29:30.020578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:30.020542    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7845-8d00f8"]
Sep 10 13:29:30.031370 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:30.031344    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7845-8d00f8"]
Sep 10 13:29:30.238855 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:30.238826    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7845-8d00f8" container="test-curl-7845-8d00f8"
Sep 10 13:29:30.673742 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:30.673710    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:31.204132 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:31.204095    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:31.244179 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:31.244139    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7845-8d00f8" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" containerName="test-curl-7845-8d00f8" containerID="cri-o://ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9" gracePeriod=2
Sep 10 13:29:31.250223 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:31.250199    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7845-8d00f8\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:31.377926 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:31.377896    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:31.408543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:31.408517    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:31.532358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:31.532334    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r926v\" (UniqueName: \"kubernetes.io/projected/5123b281-8e62-4ce3-8a89-79990684e0a6-kube-api-access-r926v\") pod \"5123b281-8e62-4ce3-8a89-79990684e0a6\" (UID: \"5123b281-8e62-4ce3-8a89-79990684e0a6\") "
Sep 10 13:29:31.534347 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:31.534323    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5123b281-8e62-4ce3-8a89-79990684e0a6-kube-api-access-r926v" (OuterVolumeSpecName: "kube-api-access-r926v") pod "5123b281-8e62-4ce3-8a89-79990684e0a6" (UID: "5123b281-8e62-4ce3-8a89-79990684e0a6"). InnerVolumeSpecName "kube-api-access-r926v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:29:31.633612 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:31.633587    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r926v\" (UniqueName: \"kubernetes.io/projected/5123b281-8e62-4ce3-8a89-79990684e0a6-kube-api-access-r926v\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:29:32.250250 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:32.250214    2551 generic.go:356] "Generic (PLEG): container finished" podID="5123b281-8e62-4ce3-8a89-79990684e0a6" containerID="e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb" exitCode=137
Sep 10 13:29:32.250742 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:32.250271    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:32.250742 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:32.250288    2551 scope.go:117] "RemoveContainer" containerID="e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb"
Sep 10 13:29:32.259253 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:32.259236    2551 scope.go:117] "RemoveContainer" containerID="e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb"
Sep 10 13:29:32.259556 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:32.259521    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb\": container with ID starting with e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb not found: ID does not exist" containerID="e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb"
Sep 10 13:29:32.259634 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:32.259566    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb"} err="failed to get container status \"e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb\": rpc error: code = NotFound desc = could not find container \"e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb\": container with ID starting with e1aa6d05046f84378a3b6beea091df53b9f8b37cfa127f6d0bf68db55410e5eb not found: ID does not exist"
Sep 10 13:29:32.261965 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:32.261945    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7845-6b7f5a\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="5123b281-8e62-4ce3-8a89-79990684e0a6" pod="opendatahub/test-curl-7845-6b7f5a"
Sep 10 13:29:33.248520 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.248496    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5123b281-8e62-4ce3-8a89-79990684e0a6" path="/var/lib/kubelet/pods/5123b281-8e62-4ce3-8a89-79990684e0a6/volumes"
Sep 10 13:29:33.417770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.417750    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:33.550231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.550208    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd5kv\" (UniqueName: \"kubernetes.io/projected/bebb00ee-8b8f-405e-acb3-586af8608193-kube-api-access-hd5kv\") pod \"bebb00ee-8b8f-405e-acb3-586af8608193\" (UID: \"bebb00ee-8b8f-405e-acb3-586af8608193\") "
Sep 10 13:29:33.552032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.552011    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bebb00ee-8b8f-405e-acb3-586af8608193-kube-api-access-hd5kv" (OuterVolumeSpecName: "kube-api-access-hd5kv") pod "bebb00ee-8b8f-405e-acb3-586af8608193" (UID: "bebb00ee-8b8f-405e-acb3-586af8608193"). InnerVolumeSpecName "kube-api-access-hd5kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:29:33.570110 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.570081    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:33.650952 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.650933    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd5kv\" (UniqueName: \"kubernetes.io/projected/bebb00ee-8b8f-405e-acb3-586af8608193-kube-api-access-hd5kv\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:29:33.678494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.678472    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:33.870208 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:33.870151    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:34.261348 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.261256    2551 generic.go:356] "Generic (PLEG): container finished" podID="bebb00ee-8b8f-405e-acb3-586af8608193" containerID="ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9" exitCode=137
Sep 10 13:29:34.261348 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.261294    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:34.261348 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.261321    2551 scope.go:117] "RemoveContainer" containerID="ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9"
Sep 10 13:29:34.270903 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.270887    2551 scope.go:117] "RemoveContainer" containerID="ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9"
Sep 10 13:29:34.271162 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:34.271135    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9\": container with ID starting with ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9 not found: ID does not exist" containerID="ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9"
Sep 10 13:29:34.271261 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.271169    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9"} err="failed to get container status \"ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9\": rpc error: code = NotFound desc = could not find container \"ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9\": container with ID starting with ce45cff249855cdbd612c858aa8193b7b38393887acedb43cb9fca9bea0fd3b9 not found: ID does not exist"
Sep 10 13:29:34.272573 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:34.272554    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7845-8d00f8\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:34.294617 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.294597    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:34.407875 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.407850    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:34.835078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.835046    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:29:34.886316 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:34.886280    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:35.247363 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:29:35.247282    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7845-8d00f8\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" pod="opendatahub/test-curl-7845-8d00f8"
Sep 10 13:29:35.249323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:35.249300    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" path="/var/lib/kubelet/pods/bebb00ee-8b8f-405e-acb3-586af8608193/volumes"
Sep 10 13:29:35.878441 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:35.878409    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:35.975426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:35.975395    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:36.261980 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.261910    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:36.749174 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.749147    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:36.860826 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.860800    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:36.944224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.944195    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"]
Sep 10 13:29:36.944712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.944692    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" containerName="test-curl-7845-8d00f8"
Sep 10 13:29:36.944781 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.944716    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" containerName="test-curl-7845-8d00f8"
Sep 10 13:29:36.944818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.944793    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bebb00ee-8b8f-405e-acb3-586af8608193" containerName="test-curl-7845-8d00f8"
Sep 10 13:29:36.949040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.949025    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:36.952162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.952143    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-af59ca\""
Sep 10 13:29:36.967051 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:36.967026    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"]
Sep 10 13:29:37.015178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.015120    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"]
Sep 10 13:29:37.018179 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.018163    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.022000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.021978    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-af59ca-dockercfg-xz4q2\""
Sep 10 13:29:37.034638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.034617    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"]
Sep 10 13:29:37.080367 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.080346    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpdgw\" (UniqueName: \"kubernetes.io/projected/68361009-7168-4835-a31f-950ed5b8c393-kube-api-access-cpdgw\") pod \"payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") " pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.080461 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.080381    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs66f\" (UniqueName: \"kubernetes.io/projected/3ed241be-e2bc-41ca-b503-efa8cd259ac7-kube-api-access-fs66f\") pod \"payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:37.080461 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.080443    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68361009-7168-4835-a31f-950ed5b8c393-plugins-config-volume\") pod \"payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") " pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.080538 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.080494    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ed241be-e2bc-41ca-b503-efa8cd259ac7-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:37.181554 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.181524    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpdgw\" (UniqueName: \"kubernetes.io/projected/68361009-7168-4835-a31f-950ed5b8c393-kube-api-access-cpdgw\") pod \"payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") " pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.181696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.181560    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs66f\" (UniqueName: \"kubernetes.io/projected/3ed241be-e2bc-41ca-b503-efa8cd259ac7-kube-api-access-fs66f\") pod \"payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:37.181696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.181596    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68361009-7168-4835-a31f-950ed5b8c393-plugins-config-volume\") pod \"payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") " pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.181696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.181635    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ed241be-e2bc-41ca-b503-efa8cd259ac7-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:37.182510 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.182487    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ed241be-e2bc-41ca-b503-efa8cd259ac7-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:37.182563 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.182517    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68361009-7168-4835-a31f-950ed5b8c393-plugins-config-volume\") pod \"payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") " pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.192389 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.192366    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs66f\" (UniqueName: \"kubernetes.io/projected/3ed241be-e2bc-41ca-b503-efa8cd259ac7-kube-api-access-fs66f\") pod \"payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:37.192869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.192847    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpdgw\" (UniqueName: \"kubernetes.io/projected/68361009-7168-4835-a31f-950ed5b8c393-kube-api-access-cpdgw\") pod \"payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") " pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.258167 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.258139    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:37.330057 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.330021    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:37.533912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.533886    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"]
Sep 10 13:29:37.536387 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:37.536361    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ed241be_e2bc_41ca_b503_efa8cd259ac7.slice/crio-9afc47756737c473d21dd81193963724d3a4c7a970708a0e68cfabcb97c4f3f5 WatchSource:0}: Error finding container 9afc47756737c473d21dd81193963724d3a4c7a970708a0e68cfabcb97c4f3f5: Status 404 returned error can't find the container with id 9afc47756737c473d21dd81193963724d3a4c7a970708a0e68cfabcb97c4f3f5
Sep 10 13:29:37.609887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.609861    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"]
Sep 10 13:29:37.611434 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:37.611409    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68361009_7168_4835_a31f_950ed5b8c393.slice/crio-d562b2392bd78d52f1bd761c170c71d978167f79c0eb23ee3f3273df76fd04fe WatchSource:0}: Error finding container d562b2392bd78d52f1bd761c170c71d978167f79c0eb23ee3f3273df76fd04fe: Status 404 returned error can't find the container with id d562b2392bd78d52f1bd761c170c71d978167f79c0eb23ee3f3273df76fd04fe
Sep 10 13:29:37.625489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.625462    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:37.694361 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:37.694337    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:38.280507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:38.280454    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct" event={"ID":"3ed241be-e2bc-41ca-b503-efa8cd259ac7","Type":"ContainerStarted","Data":"9afc47756737c473d21dd81193963724d3a4c7a970708a0e68cfabcb97c4f3f5"}
Sep 10 13:29:38.281851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:38.281824    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9" event={"ID":"68361009-7168-4835-a31f-950ed5b8c393","Type":"ContainerStarted","Data":"d562b2392bd78d52f1bd761c170c71d978167f79c0eb23ee3f3273df76fd04fe"}
Sep 10 13:29:38.844677 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:38.844630    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:38.945788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:38.945751    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:39.043137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.043080    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:39.382834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.382770    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:39.484128 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.484095    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:39.643463 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.643274    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"]
Sep 10 13:29:39.648645 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.648619    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:39.653452 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.653428    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d2fb9c2a-dockercfg-cmcl8\""
Sep 10 13:29:39.668243 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.668221    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d2fb9c2a\""
Sep 10 13:29:39.685530 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.685481    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"]
Sep 10 13:29:39.805203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.805169    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:39.805203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.805209    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjjbv\" (UniqueName: \"kubernetes.io/projected/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-kube-api-access-mjjbv\") pod \"payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:39.906317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.906223    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:39.906317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.906275    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjjbv\" (UniqueName: \"kubernetes.io/projected/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-kube-api-access-mjjbv\") pod \"payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:39.907033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.907009    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:39.915867 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.915841    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjjbv\" (UniqueName: \"kubernetes.io/projected/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-kube-api-access-mjjbv\") pod \"payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:39.962274 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:39.962225    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:40.294181 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:40.294062    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct" event={"ID":"3ed241be-e2bc-41ca-b503-efa8cd259ac7","Type":"ContainerStarted","Data":"41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f"}
Sep 10 13:29:40.294181 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:40.294123    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:40.295893 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:40.295867    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9" event={"ID":"68361009-7168-4835-a31f-950ed5b8c393","Type":"ContainerStarted","Data":"ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452"}
Sep 10 13:29:40.296011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:40.295984    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:40.324940 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:40.324895    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct" podStartSLOduration=1.657509799 podStartE2EDuration="4.324879821s" totalImagesPullingTime="2.667370022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:40.324012788 +0000 UTC m=+745.535206045" watchObservedRunningTime="2026-09-10 13:29:40.324879821 +0000 UTC m=+745.536073078"
Sep 10 13:29:40.349968 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:40.349926    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9" podStartSLOduration=1.7653419000000001 podStartE2EDuration="4.349912098s" totalImagesPullingTime="2.584570198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:40.346706152 +0000 UTC m=+745.557899410" watchObservedRunningTime="2026-09-10 13:29:40.349912098 +0000 UTC m=+745.561105387"
Sep 10 13:29:40.408062 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:40.408026    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"]
Sep 10 13:29:40.409186 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:40.409160    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894446e8_1d7e_43fb_9079_bdf81c4cb9d6.slice/crio-66176a2248c0d435da1cd7ea0307623944db28ea483fbbb3755ed04a9b326d9e WatchSource:0}: Error finding container 66176a2248c0d435da1cd7ea0307623944db28ea483fbbb3755ed04a9b326d9e: Status 404 returned error can't find the container with id 66176a2248c0d435da1cd7ea0307623944db28ea483fbbb3755ed04a9b326d9e
Sep 10 13:29:41.302713 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:41.302653    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5" event={"ID":"894446e8-1d7e-43fb-9079-bdf81c4cb9d6","Type":"ContainerStarted","Data":"00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d"}
Sep 10 13:29:41.302914 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:41.302730    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5" event={"ID":"894446e8-1d7e-43fb-9079-bdf81c4cb9d6","Type":"ContainerStarted","Data":"66176a2248c0d435da1cd7ea0307623944db28ea483fbbb3755ed04a9b326d9e"}
Sep 10 13:29:41.327341 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:41.327291    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5" podStartSLOduration=2.327276383 podStartE2EDuration="2.327276383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:41.32589271 +0000 UTC m=+746.537085969" watchObservedRunningTime="2026-09-10 13:29:41.327276383 +0000 UTC m=+746.538469639"
Sep 10 13:29:41.449178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:41.449149    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:41.603284 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:41.603207    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:41.710180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:41.710153    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:41.801287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:41.801260    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:42.307353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:42.307330    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:44.669115 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:44.669076    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:45.894342 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:45.894306    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:46.892771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:46.892745    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:47.039509 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:47.039473    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:47.131474 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:47.131439    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:48.762353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:48.762321    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:50.219325 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.219287    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"]
Sep 10 13:29:50.222698 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.222682    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.225847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.225823    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-5995df69-kserve-self-signed-certs\""
Sep 10 13:29:50.232282 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.232264    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"]
Sep 10 13:29:50.396938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.396898    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-tmp-dir\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.396938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.396947    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2ed8d28-804b-4824-8440-d627a2b6d24c-tls-certs\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.397416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.397006    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmq6m\" (UniqueName: \"kubernetes.io/projected/c2ed8d28-804b-4824-8440-d627a2b6d24c-kube-api-access-hmq6m\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.397416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.397041    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-model-cache\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.397416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.397073    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-dshm\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.397416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.397234    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-home\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.497814 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.497744    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-home\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.497814 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.497787    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-tmp-dir\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.497814 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.497809    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2ed8d28-804b-4824-8440-d627a2b6d24c-tls-certs\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.498031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.497913    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmq6m\" (UniqueName: \"kubernetes.io/projected/c2ed8d28-804b-4824-8440-d627a2b6d24c-kube-api-access-hmq6m\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.498031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.497945    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-model-cache\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.498031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.497974    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-dshm\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.498238 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.498215    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-tmp-dir\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.498238 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.498232    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-home\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.498367 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.498278    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-model-cache\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.499775 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.499749    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:50.500238 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.500214    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-dshm\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.500327 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.500299    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2ed8d28-804b-4824-8440-d627a2b6d24c-tls-certs\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.508299 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.508277    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmq6m\" (UniqueName: \"kubernetes.io/projected/c2ed8d28-804b-4824-8440-d627a2b6d24c-kube-api-access-hmq6m\") pod \"e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") " pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.532430 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.532409    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:29:50.806393 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:50.806364    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"]
Sep 10 13:29:50.808895 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:50.808872    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2ed8d28_804b_4824_8440_d627a2b6d24c.slice/crio-b1262d860eeee4cdd84cba6f2b7b16e42e289a45b523454bd79791fd9182f863 WatchSource:0}: Error finding container b1262d860eeee4cdd84cba6f2b7b16e42e289a45b523454bd79791fd9182f863: Status 404 returned error can't find the container with id b1262d860eeee4cdd84cba6f2b7b16e42e289a45b523454bd79791fd9182f863
Sep 10 13:29:51.213230 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:51.213148    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:51.305268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:51.305238    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:29:51.305681 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:51.305304    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:29:51.353042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:51.353008    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr" event={"ID":"c2ed8d28-804b-4824-8440-d627a2b6d24c","Type":"ContainerStarted","Data":"8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b"}
Sep 10 13:29:51.353270 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:51.353190    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr" event={"ID":"c2ed8d28-804b-4824-8440-d627a2b6d24c","Type":"ContainerStarted","Data":"b1262d860eeee4cdd84cba6f2b7b16e42e289a45b523454bd79791fd9182f863"}
Sep 10 13:29:51.383431 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:51.383374    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr" podStartSLOduration=1.383354771 podStartE2EDuration="1.383354771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:51.380654945 +0000 UTC m=+756.591848203" watchObservedRunningTime="2026-09-10 13:29:51.383354771 +0000 UTC m=+756.594548030"
Sep 10 13:29:51.503810 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:51.503745    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:52.218174 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:52.218137    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:29:52.284607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:52.284555    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:52.384426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:52.384397    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:53.320098 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:53.320070    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:29:54.588047 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:54.588011    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:54.739530 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:54.739494    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:54.834351 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:54.834320    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:55.060275 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:55.060246    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:55.744206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:55.744176    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:55.843543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:55.843505    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:57.749536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.749503    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"]
Sep 10 13:29:57.753238 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.753222    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:57.791185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.791165    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-6beee4-dockercfg-kzh56\""
Sep 10 13:29:57.791587 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.791562    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-6beee4\""
Sep 10 13:29:57.801109 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.801086    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"]
Sep 10 13:29:57.804383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.804369    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:57.818228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.818208    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"]
Sep 10 13:29:57.863447 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.863422    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw62n\" (UniqueName: \"kubernetes.io/projected/476d37a6-a48d-4af5-9f07-81df1c767d86-kube-api-access-zw62n\") pod \"payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:57.863531 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.863513    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/476d37a6-a48d-4af5-9f07-81df1c767d86-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:57.871369 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.871345    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"]
Sep 10 13:29:57.947996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.947976    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:57.963964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.963938    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54bdz\" (UniqueName: \"kubernetes.io/projected/eca57a56-67bf-493e-aa29-b0a3c186b691-kube-api-access-54bdz\") pod \"payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:57.964052 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.963978    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/476d37a6-a48d-4af5-9f07-81df1c767d86-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:57.964052 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.964024    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca57a56-67bf-493e-aa29-b0a3c186b691-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:57.964138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.964114    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw62n\" (UniqueName: \"kubernetes.io/projected/476d37a6-a48d-4af5-9f07-81df1c767d86-kube-api-access-zw62n\") pod \"payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:57.964559 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.964543    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/476d37a6-a48d-4af5-9f07-81df1c767d86-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:57.984350 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:57.984329    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw62n\" (UniqueName: \"kubernetes.io/projected/476d37a6-a48d-4af5-9f07-81df1c767d86-kube-api-access-zw62n\") pod \"payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:58.046151 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.046118    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:58.063830 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.063807    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:58.065071 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.065051    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca57a56-67bf-493e-aa29-b0a3c186b691-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:58.065155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.065144    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54bdz\" (UniqueName: \"kubernetes.io/projected/eca57a56-67bf-493e-aa29-b0a3c186b691-kube-api-access-54bdz\") pod \"payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:58.065739 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.065722    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca57a56-67bf-493e-aa29-b0a3c186b691-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:58.077907 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.077886    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54bdz\" (UniqueName: \"kubernetes.io/projected/eca57a56-67bf-493e-aa29-b0a3c186b691-kube-api-access-54bdz\") pod \"payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:58.113489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.113280    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:58.346145 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.344183    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"]
Sep 10 13:29:58.350157 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:58.350123    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476d37a6_a48d_4af5_9f07_81df1c767d86.slice/crio-beb265d5f71c6ed9c9ea69846475c15c645f3dc3cf2387330e4ddb39edd1ba15 WatchSource:0}: Error finding container beb265d5f71c6ed9c9ea69846475c15c645f3dc3cf2387330e4ddb39edd1ba15: Status 404 returned error can't find the container with id beb265d5f71c6ed9c9ea69846475c15c645f3dc3cf2387330e4ddb39edd1ba15
Sep 10 13:29:58.386960 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.386932    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn" event={"ID":"476d37a6-a48d-4af5-9f07-81df1c767d86","Type":"ContainerStarted","Data":"beb265d5f71c6ed9c9ea69846475c15c645f3dc3cf2387330e4ddb39edd1ba15"}
Sep 10 13:29:58.409116 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.409096    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"]
Sep 10 13:29:58.411187 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:29:58.411159    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeca57a56_67bf_493e_aa29_b0a3c186b691.slice/crio-2c74404e1485f336d6decc246289598cfe290f4d2bf0237f6f36b9463da794fb WatchSource:0}: Error finding container 2c74404e1485f336d6decc246289598cfe290f4d2bf0237f6f36b9463da794fb: Status 404 returned error can't find the container with id 2c74404e1485f336d6decc246289598cfe290f4d2bf0237f6f36b9463da794fb
Sep 10 13:29:58.497630 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:58.497598    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:29:59.393227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.393191    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn" event={"ID":"476d37a6-a48d-4af5-9f07-81df1c767d86","Type":"ContainerStarted","Data":"fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566"}
Sep 10 13:29:59.393701 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.393308    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:29:59.394982 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.394961    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b" event={"ID":"eca57a56-67bf-493e-aa29-b0a3c186b691","Type":"ContainerStarted","Data":"2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364"}
Sep 10 13:29:59.395104 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.394997    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b" event={"ID":"eca57a56-67bf-493e-aa29-b0a3c186b691","Type":"ContainerStarted","Data":"2c74404e1485f336d6decc246289598cfe290f4d2bf0237f6f36b9463da794fb"}
Sep 10 13:29:59.395104 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.395032    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:29:59.418281 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.418237    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn" podStartSLOduration=2.4182168170000002 podStartE2EDuration="2.418216817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:59.416597731 +0000 UTC m=+764.627790990" watchObservedRunningTime="2026-09-10 13:29:59.418216817 +0000 UTC m=+764.629410074"
Sep 10 13:29:59.443998 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.443958    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b" podStartSLOduration=2.443945111 podStartE2EDuration="2.443945111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:29:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:29:59.443151205 +0000 UTC m=+764.654344453" watchObservedRunningTime="2026-09-10 13:29:59.443945111 +0000 UTC m=+764.655138367"
Sep 10 13:29:59.870359 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:29:59.870320    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:00.229065 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.228988    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"]
Sep 10 13:30:00.232220 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.232201    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"
Sep 10 13:30:00.238760 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.238733    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl"]
Sep 10 13:30:00.242793 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.242771    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:30:00.243289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.243268    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl"
Sep 10 13:30:00.244735 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.244718    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"
Sep 10 13:30:00.253712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.253648    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"]
Sep 10 13:30:00.260256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.260235    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"
Sep 10 13:30:00.276234 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.276215    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"]
Sep 10 13:30:00.276419 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.276404    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"]
Sep 10 13:30:00.284197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.284173    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl"]
Sep 10 13:30:00.327490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.325291    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl"
Sep 10 13:30:00.345365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.345342    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"
Sep 10 13:30:00.537521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.536742    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:30:00.537521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.537473    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:30:00.556636 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.556605    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:30:00.599482 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.599453    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"]
Sep 10 13:30:00.601304 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:30:00.601280    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90cc955d_900d_4511_b323_5e176bfa8240.slice/crio-5bc2fb1cc748df11d04f1492acdaf2664b6de83a605772202f12e71b23e360b2 WatchSource:0}: Error finding container 5bc2fb1cc748df11d04f1492acdaf2664b6de83a605772202f12e71b23e360b2: Status 404 returned error can't find the container with id 5bc2fb1cc748df11d04f1492acdaf2664b6de83a605772202f12e71b23e360b2
Sep 10 13:30:00.647150 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.647119    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl"]
Sep 10 13:30:00.651217 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:30:00.651188    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e41435_71ff_4706_8d6f_dc51e2fb117a.slice/crio-7a258f248104e20325ea6c57f8b1bcd2db7daf73c4b1e9c0116b1142ee8d7e7c WatchSource:0}: Error finding container 7a258f248104e20325ea6c57f8b1bcd2db7daf73c4b1e9c0116b1142ee8d7e7c: Status 404 returned error can't find the container with id 7a258f248104e20325ea6c57f8b1bcd2db7daf73c4b1e9c0116b1142ee8d7e7c
Sep 10 13:30:00.673963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:00.673943    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"]
Sep 10 13:30:00.675275 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:30:00.675243    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54a66261_58f8_4cb0_bf15_2ebe961e11a5.slice/crio-2b6bdbf5e8cf4f27cb452f025dccf3b8c05e0210c55b383798c25cf9e7df0bbb WatchSource:0}: Error finding container 2b6bdbf5e8cf4f27cb452f025dccf3b8c05e0210c55b383798c25cf9e7df0bbb: Status 404 returned error can't find the container with id 2b6bdbf5e8cf4f27cb452f025dccf3b8c05e0210c55b383798c25cf9e7df0bbb
Sep 10 13:30:01.412472 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:01.412434    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl" event={"ID":"02e41435-71ff-4706-8d6f-dc51e2fb117a","Type":"ContainerStarted","Data":"7a258f248104e20325ea6c57f8b1bcd2db7daf73c4b1e9c0116b1142ee8d7e7c"}
Sep 10 13:30:01.413639 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:01.413612    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm" event={"ID":"90cc955d-900d-4511-b323-5e176bfa8240","Type":"ContainerStarted","Data":"5bc2fb1cc748df11d04f1492acdaf2664b6de83a605772202f12e71b23e360b2"}
Sep 10 13:30:01.414742 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:01.414717    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9" event={"ID":"54a66261-58f8-4cb0-bf15-2ebe961e11a5","Type":"ContainerStarted","Data":"2b6bdbf5e8cf4f27cb452f025dccf3b8c05e0210c55b383798c25cf9e7df0bbb"}
Sep 10 13:30:01.426044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:01.426022    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:30:01.638300 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:01.638264    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:03.687434 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:03.687406    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:04.430292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.430215    2551 generic.go:356] "Generic (PLEG): container finished" podID="54a66261-58f8-4cb0-bf15-2ebe961e11a5" containerID="15cd7c2e0f4ce2d645514578208936ec9fffdcc6c05cc1c41116f26f305230d3" exitCode=0
Sep 10 13:30:04.430422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.430291    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9" event={"ID":"54a66261-58f8-4cb0-bf15-2ebe961e11a5","Type":"ContainerDied","Data":"15cd7c2e0f4ce2d645514578208936ec9fffdcc6c05cc1c41116f26f305230d3"}
Sep 10 13:30:04.431820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.431800    2551 generic.go:356] "Generic (PLEG): container finished" podID="02e41435-71ff-4706-8d6f-dc51e2fb117a" containerID="f7b33e34aaad7d6f902c4e21e18cac8182fc059b6210eb9e67e1dcb3d0837691" exitCode=0
Sep 10 13:30:04.431929 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.431858    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl" event={"ID":"02e41435-71ff-4706-8d6f-dc51e2fb117a","Type":"ContainerDied","Data":"f7b33e34aaad7d6f902c4e21e18cac8182fc059b6210eb9e67e1dcb3d0837691"}
Sep 10 13:30:04.433225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.433206    2551 generic.go:356] "Generic (PLEG): container finished" podID="90cc955d-900d-4511-b323-5e176bfa8240" containerID="b8a231706b52e526932a8340d238ec0151fceb1e9fa93907de87041247a0ad6a" exitCode=0
Sep 10 13:30:04.433283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.433238    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm" event={"ID":"90cc955d-900d-4511-b323-5e176bfa8240","Type":"ContainerDied","Data":"b8a231706b52e526932a8340d238ec0151fceb1e9fa93907de87041247a0ad6a"}
Sep 10 13:30:04.515866 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.515839    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:04.657843 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:04.657796    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:05.574404 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:05.574377    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl"
Sep 10 13:30:05.618122 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:05.618100    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"
Sep 10 13:30:05.621246 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:05.621227    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"
Sep 10 13:30:06.443784 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.443756    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"
Sep 10 13:30:06.443951 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.443782    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9" event={"ID":"54a66261-58f8-4cb0-bf15-2ebe961e11a5","Type":"ContainerDied","Data":"2b6bdbf5e8cf4f27cb452f025dccf3b8c05e0210c55b383798c25cf9e7df0bbb"}
Sep 10 13:30:06.443951 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.443831    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b6bdbf5e8cf4f27cb452f025dccf3b8c05e0210c55b383798c25cf9e7df0bbb"
Sep 10 13:30:06.445115 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.445082    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl"
Sep 10 13:30:06.445265 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.445115    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817450-jhxbl" event={"ID":"02e41435-71ff-4706-8d6f-dc51e2fb117a","Type":"ContainerDied","Data":"7a258f248104e20325ea6c57f8b1bcd2db7daf73c4b1e9c0116b1142ee8d7e7c"}
Sep 10 13:30:06.445265 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.445151    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a258f248104e20325ea6c57f8b1bcd2db7daf73c4b1e9c0116b1142ee8d7e7c"
Sep 10 13:30:06.446418 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.446397    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm" event={"ID":"90cc955d-900d-4511-b323-5e176bfa8240","Type":"ContainerDied","Data":"5bc2fb1cc748df11d04f1492acdaf2664b6de83a605772202f12e71b23e360b2"}
Sep 10 13:30:06.446418 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.446417    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"
Sep 10 13:30:06.446604 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:06.446431    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bc2fb1cc748df11d04f1492acdaf2664b6de83a605772202f12e71b23e360b2"
Sep 10 13:30:08.462938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.462909    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"]
Sep 10 13:30:08.463289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463238    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54a66261-58f8-4cb0-bf15-2ebe961e11a5" containerName="cleanup"
Sep 10 13:30:08.463289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463248    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="54a66261-58f8-4cb0-bf15-2ebe961e11a5" containerName="cleanup"
Sep 10 13:30:08.463289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463256    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90cc955d-900d-4511-b323-5e176bfa8240" containerName="cleanup"
Sep 10 13:30:08.463289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463262    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="90cc955d-900d-4511-b323-5e176bfa8240" containerName="cleanup"
Sep 10 13:30:08.463289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463269    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02e41435-71ff-4706-8d6f-dc51e2fb117a" containerName="cleanup"
Sep 10 13:30:08.463289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463274    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="02e41435-71ff-4706-8d6f-dc51e2fb117a" containerName="cleanup"
Sep 10 13:30:08.463483 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463336    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="90cc955d-900d-4511-b323-5e176bfa8240" containerName="cleanup"
Sep 10 13:30:08.463483 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463344    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="54a66261-58f8-4cb0-bf15-2ebe961e11a5" containerName="cleanup"
Sep 10 13:30:08.463483 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.463352    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="02e41435-71ff-4706-8d6f-dc51e2fb117a" containerName="cleanup"
Sep 10 13:30:08.466349 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.466327    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"
Sep 10 13:30:08.469645 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.469624    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:30:08.469780 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.469718    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:30:08.476461 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.476439    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"]
Sep 10 13:30:08.556718 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.556685    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcd78b6-a429-43ea-9113-e483f105ed6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2\" (UID: \"2dcd78b6-a429-43ea-9113-e483f105ed6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"
Sep 10 13:30:08.657487 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.657458    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcd78b6-a429-43ea-9113-e483f105ed6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2\" (UID: \"2dcd78b6-a429-43ea-9113-e483f105ed6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"
Sep 10 13:30:08.658100 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.658075    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcd78b6-a429-43ea-9113-e483f105ed6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2\" (UID: \"2dcd78b6-a429-43ea-9113-e483f105ed6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"
Sep 10 13:30:08.778924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:08.778901    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"
Sep 10 13:30:09.058379 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:09.058354    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"]
Sep 10 13:30:09.060932 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:30:09.060900    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dcd78b6_a429_43ea_9113_e483f105ed6b.slice/crio-45d0a9469c21b116d1c1db3a3edaea2fd759bfa8235a4e0007ee5a51ae2bfb96 WatchSource:0}: Error finding container 45d0a9469c21b116d1c1db3a3edaea2fd759bfa8235a4e0007ee5a51ae2bfb96: Status 404 returned error can't find the container with id 45d0a9469c21b116d1c1db3a3edaea2fd759bfa8235a4e0007ee5a51ae2bfb96
Sep 10 13:30:09.463285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:09.463257    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2" event={"ID":"2dcd78b6-a429-43ea-9113-e483f105ed6b","Type":"ContainerStarted","Data":"aa920932ae73b26f9149f4b5d818b9826c89231210f46c45367b741ae041d285"}
Sep 10 13:30:09.463584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:09.463310    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2" event={"ID":"2dcd78b6-a429-43ea-9113-e483f105ed6b","Type":"ContainerStarted","Data":"45d0a9469c21b116d1c1db3a3edaea2fd759bfa8235a4e0007ee5a51ae2bfb96"}
Sep 10 13:30:09.486991 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:09.486946    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2" podStartSLOduration=1.48692999 podStartE2EDuration="1.48692999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:30:09.485658121 +0000 UTC m=+774.696851377" watchObservedRunningTime="2026-09-10 13:30:09.48692999 +0000 UTC m=+774.698123294"
Sep 10 13:30:10.092326 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:10.092294    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:30:10.408869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:10.408783    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:30:10.409066 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:10.408876    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:30:10.471910 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:10.471878    2551 generic.go:356] "Generic (PLEG): container finished" podID="2dcd78b6-a429-43ea-9113-e483f105ed6b" containerID="aa920932ae73b26f9149f4b5d818b9826c89231210f46c45367b741ae041d285" exitCode=0
Sep 10 13:30:10.472341 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:10.471953    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2" event={"ID":"2dcd78b6-a429-43ea-9113-e483f105ed6b","Type":"ContainerDied","Data":"aa920932ae73b26f9149f4b5d818b9826c89231210f46c45367b741ae041d285"}
Sep 10 13:30:11.602503 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:11.602483    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"
Sep 10 13:30:11.683063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:11.683036    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcd78b6-a429-43ea-9113-e483f105ed6b-maas-api-service-ca\") pod \"2dcd78b6-a429-43ea-9113-e483f105ed6b\" (UID: \"2dcd78b6-a429-43ea-9113-e483f105ed6b\") "
Sep 10 13:30:11.683388 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:11.683365    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dcd78b6-a429-43ea-9113-e483f105ed6b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2dcd78b6-a429-43ea-9113-e483f105ed6b" (UID: "2dcd78b6-a429-43ea-9113-e483f105ed6b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:30:11.784532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:11.784510    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2dcd78b6-a429-43ea-9113-e483f105ed6b-maas-api-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:30:12.483599 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:12.483565    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2" event={"ID":"2dcd78b6-a429-43ea-9113-e483f105ed6b","Type":"ContainerDied","Data":"45d0a9469c21b116d1c1db3a3edaea2fd759bfa8235a4e0007ee5a51ae2bfb96"}
Sep 10 13:30:12.483599 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:12.483592    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"
Sep 10 13:30:12.483829 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:12.483621    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45d0a9469c21b116d1c1db3a3edaea2fd759bfa8235a4e0007ee5a51ae2bfb96"
Sep 10 13:30:18.317937 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:18.317898    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:19.024201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.024169    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:19.039370 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.039340    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"]
Sep 10 13:30:19.047522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.047495    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d2fb9c2a-29817450-l6prm"]
Sep 10 13:30:19.092800 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.092771    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"]
Sep 10 13:30:19.093227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.093173    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5" podUID="894446e8-1d7e-43fb-9079-bdf81c4cb9d6" containerName="payload-pre-processing" containerID="cri-o://00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d" gracePeriod=30
Sep 10 13:30:19.121063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.121040    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:19.189799 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.189755    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:19.248908 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.248878    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90cc955d-900d-4511-b323-5e176bfa8240" path="/var/lib/kubelet/pods/90cc955d-900d-4511-b323-5e176bfa8240/volumes"
Sep 10 13:30:19.305150 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.305071    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:19.340868 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.340839    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:30:19.341772 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.341479    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"]
Sep 10 13:30:19.343914 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.343893    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d2fb9c2a-sz2j2"]
Sep 10 13:30:19.366680 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.366647    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:19.442056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.442032    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjjbv\" (UniqueName: \"kubernetes.io/projected/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-kube-api-access-mjjbv\") pod \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") "
Sep 10 13:30:19.442157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.442099    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-plugins-config-volume\") pod \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\" (UID: \"894446e8-1d7e-43fb-9079-bdf81c4cb9d6\") "
Sep 10 13:30:19.442484 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.442463    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "894446e8-1d7e-43fb-9079-bdf81c4cb9d6" (UID: "894446e8-1d7e-43fb-9079-bdf81c4cb9d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:30:19.444254 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.444222    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-kube-api-access-mjjbv" (OuterVolumeSpecName: "kube-api-access-mjjbv") pod "894446e8-1d7e-43fb-9079-bdf81c4cb9d6" (UID: "894446e8-1d7e-43fb-9079-bdf81c4cb9d6"). InnerVolumeSpecName "kube-api-access-mjjbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:30:19.516763 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.516730    2551 generic.go:356] "Generic (PLEG): container finished" podID="894446e8-1d7e-43fb-9079-bdf81c4cb9d6" containerID="00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d" exitCode=0
Sep 10 13:30:19.516930 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.516774    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"
Sep 10 13:30:19.516930 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.516819    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5" event={"ID":"894446e8-1d7e-43fb-9079-bdf81c4cb9d6","Type":"ContainerDied","Data":"00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d"}
Sep 10 13:30:19.516930 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.516878    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5" event={"ID":"894446e8-1d7e-43fb-9079-bdf81c4cb9d6","Type":"ContainerDied","Data":"66176a2248c0d435da1cd7ea0307623944db28ea483fbbb3755ed04a9b326d9e"}
Sep 10 13:30:19.516930 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.516907    2551 scope.go:117] "RemoveContainer" containerID="00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d"
Sep 10 13:30:19.525485 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.525468    2551 scope.go:117] "RemoveContainer" containerID="00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d"
Sep 10 13:30:19.525793 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:30:19.525770    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d\": container with ID starting with 00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d not found: ID does not exist" containerID="00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d"
Sep 10 13:30:19.525855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.525803    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d"} err="failed to get container status \"00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d\": rpc error: code = NotFound desc = could not find container \"00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d\": container with ID starting with 00e2904dc539cc6f04a7d3b7bbb68d73361a4ae119d4eac8daf294f9466da09d not found: ID does not exist"
Sep 10 13:30:19.543010 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.542985    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:30:19.543010 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.543006    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjjbv\" (UniqueName: \"kubernetes.io/projected/894446e8-1d7e-43fb-9079-bdf81c4cb9d6-kube-api-access-mjjbv\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:30:19.544514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.544496    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"]
Sep 10 13:30:19.556152 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:19.556103    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d2fb9c2a-c45fdb4bb-4v2t5"]
Sep 10 13:30:20.397568 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:20.397531    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:20.534488 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:20.534461    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:20.665839 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:20.665768    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:21.218937 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:21.218907    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:21.248445 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:21.248414    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dcd78b6-a429-43ea-9113-e483f105ed6b" path="/var/lib/kubelet/pods/2dcd78b6-a429-43ea-9113-e483f105ed6b/volumes"
Sep 10 13:30:21.248775 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:21.248763    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="894446e8-1d7e-43fb-9079-bdf81c4cb9d6" path="/var/lib/kubelet/pods/894446e8-1d7e-43fb-9079-bdf81c4cb9d6/volumes"
Sep 10 13:30:22.504479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:22.504448    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:22.603845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:22.603817    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:25.606089 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:25.606061    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:30:27.436632 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:27.436594    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:27.543450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:27.543418    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:27.686038 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:27.686002    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:27.825410 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:27.825375    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:28.323287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:28.323252    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:28.424294 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:28.424263    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:28.500708 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:28.500680    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:39.021025 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.020950    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"]
Sep 10 13:30:39.021422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.021330    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dcd78b6-a429-43ea-9113-e483f105ed6b" containerName="revoke-keys"
Sep 10 13:30:39.021422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.021343    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dcd78b6-a429-43ea-9113-e483f105ed6b" containerName="revoke-keys"
Sep 10 13:30:39.021422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.021376    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="894446e8-1d7e-43fb-9079-bdf81c4cb9d6" containerName="payload-pre-processing"
Sep 10 13:30:39.021422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.021382    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="894446e8-1d7e-43fb-9079-bdf81c4cb9d6" containerName="payload-pre-processing"
Sep 10 13:30:39.021584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.021446    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dcd78b6-a429-43ea-9113-e483f105ed6b" containerName="revoke-keys"
Sep 10 13:30:39.021584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.021454    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="894446e8-1d7e-43fb-9079-bdf81c4cb9d6" containerName="payload-pre-processing"
Sep 10 13:30:39.031634 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.031609    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.034868 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.034841    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-5995df69-kserve-self-signed-certs\""
Sep 10 13:30:39.038125 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.038101    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"]
Sep 10 13:30:39.043298 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.043278    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:39.209515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.209483    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-tmp-dir\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.209515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.209514    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-home\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.209734 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.209533    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8234dda6-0e9c-47f1-93a0-12ebb265580e-tls-certs\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.209734 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.209629    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww274\" (UniqueName: \"kubernetes.io/projected/8234dda6-0e9c-47f1-93a0-12ebb265580e-kube-api-access-ww274\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.209734 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.209695    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-dshm\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.209734 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.209720    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-model-cache\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310326 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310264    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-tmp-dir\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310326 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310295    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-home\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310326 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310310    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8234dda6-0e9c-47f1-93a0-12ebb265580e-tls-certs\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310370    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww274\" (UniqueName: \"kubernetes.io/projected/8234dda6-0e9c-47f1-93a0-12ebb265580e-kube-api-access-ww274\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310415    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-dshm\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310441    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-model-cache\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310716 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310647    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-home\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310779 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310712    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-tmp-dir\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.310822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.310796    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-model-cache\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.312548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.312527    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-dshm\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.312816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.312799    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8234dda6-0e9c-47f1-93a0-12ebb265580e-tls-certs\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.320289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.320254    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww274\" (UniqueName: \"kubernetes.io/projected/8234dda6-0e9c-47f1-93a0-12ebb265580e-kube-api-access-ww274\") pod \"e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") " pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.341061 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.341040    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:39.617697 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:39.617674    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"]
Sep 10 13:30:39.620053 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:30:39.620024    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8234dda6_0e9c_47f1_93a0_12ebb265580e.slice/crio-662c79ef325862a32563949bdc4cabed97101f4c7e2a11ef05048ad683efa008 WatchSource:0}: Error finding container 662c79ef325862a32563949bdc4cabed97101f4c7e2a11ef05048ad683efa008: Status 404 returned error can't find the container with id 662c79ef325862a32563949bdc4cabed97101f4c7e2a11ef05048ad683efa008
Sep 10 13:30:40.616506 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:40.616469    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h" event={"ID":"8234dda6-0e9c-47f1-93a0-12ebb265580e","Type":"ContainerStarted","Data":"38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5"}
Sep 10 13:30:40.616898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:40.616514    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h" event={"ID":"8234dda6-0e9c-47f1-93a0-12ebb265580e","Type":"ContainerStarted","Data":"662c79ef325862a32563949bdc4cabed97101f4c7e2a11ef05048ad683efa008"}
Sep 10 13:30:40.640556 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:40.640514    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h" podStartSLOduration=1.640499103 podStartE2EDuration="1.640499103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:30:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:30:40.639151029 +0000 UTC m=+805.850344286" watchObservedRunningTime="2026-09-10 13:30:40.640499103 +0000 UTC m=+805.851692359"
Sep 10 13:30:41.388888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:41.388858    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:30:49.341376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:49.341338    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:49.341765 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:49.341386    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:49.353914 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:49.353892    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:49.679405 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:49.679333    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:30:52.135486 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.135451    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"]
Sep 10 13:30:52.138396 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.138378    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.143608 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.143584    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-af59ca\"/\"autores-model-af59ca-kserve-self-signed-certs\""
Sep 10 13:30:52.145421 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.145396    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-af59ca\"/\"kube-root-ca.crt\""
Sep 10 13:30:52.145421 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.145413    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-af59ca\"/\"default-dockercfg-hhjdm\""
Sep 10 13:30:52.145603 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.145588    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-af59ca\"/\"openshift-service-ca.crt\""
Sep 10 13:30:52.158402 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.158382    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"]
Sep 10 13:30:52.216773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.216745    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-home\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.216773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.216777    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-model-cache\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.216964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.216802    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2d4h\" (UniqueName: \"kubernetes.io/projected/f072cce5-1269-4450-9e55-3ae092f1a740-kube-api-access-w2d4h\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.216964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.216872    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-tmp-dir\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.216964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.216926    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-dshm\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.217076 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.216981    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f072cce5-1269-4450-9e55-3ae092f1a740-tls-certs\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.256077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.256045    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:30:52.318012 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.317987    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-dshm\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318159 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318061    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f072cce5-1269-4450-9e55-3ae092f1a740-tls-certs\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318159 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318133    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-home\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318160    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-model-cache\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318197    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2d4h\" (UniqueName: \"kubernetes.io/projected/f072cce5-1269-4450-9e55-3ae092f1a740-kube-api-access-w2d4h\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318231    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-tmp-dir\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318508 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318487    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-home\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318571 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318554    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-tmp-dir\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.318617 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.318580    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-model-cache\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.320119 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.320101    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-dshm\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.320500 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.320484    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f072cce5-1269-4450-9e55-3ae092f1a740-tls-certs\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.328987 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.328967    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2d4h\" (UniqueName: \"kubernetes.io/projected/f072cce5-1269-4450-9e55-3ae092f1a740-kube-api-access-w2d4h\") pod \"autores-model-af59ca-kserve-5885dd79c5-2qxpr\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") " pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.447826 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.447764    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:30:52.726982 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:52.726952    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"]
Sep 10 13:30:52.729440 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:30:52.729413    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf072cce5_1269_4450_9e55_3ae092f1a740.slice/crio-f60185d4e37e9b3464aeeba7ee8ed1dc9c8a3a395bfea62810742967b2aa21a8 WatchSource:0}: Error finding container f60185d4e37e9b3464aeeba7ee8ed1dc9c8a3a395bfea62810742967b2aa21a8: Status 404 returned error can't find the container with id f60185d4e37e9b3464aeeba7ee8ed1dc9c8a3a395bfea62810742967b2aa21a8
Sep 10 13:30:53.688946 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:53.688914    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr" event={"ID":"f072cce5-1269-4450-9e55-3ae092f1a740","Type":"ContainerStarted","Data":"dbb541190c9a0ee2df9da54e7fe98d16cceef658569092689fe2dc8e816e56eb"}
Sep 10 13:30:53.689311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:53.688956    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr" event={"ID":"f072cce5-1269-4450-9e55-3ae092f1a740","Type":"ContainerStarted","Data":"f60185d4e37e9b3464aeeba7ee8ed1dc9c8a3a395bfea62810742967b2aa21a8"}
Sep 10 13:30:53.721409 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:53.721360    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr" podStartSLOduration=1.7213455720000002 podStartE2EDuration="1.721345572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:30:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:30:53.720553214 +0000 UTC m=+818.931746471" watchObservedRunningTime="2026-09-10 13:30:53.721345572 +0000 UTC m=+818.932538828"
Sep 10 13:30:59.590182 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:30:59.590130    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:00.251028 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:00.250992    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:31:00.483679 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:00.483619    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:00.702020 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:00.701991    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:00.841748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:00.841716    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:01.287736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:01.287703    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:01.742388 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:01.742283    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:01.844542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:01.844513    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:02.448331 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:02.448302    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:31:02.448508 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:02.448343    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:31:02.460671 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:02.460648    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:31:02.743190 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:02.743127    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:31:04.416891 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:04.416859    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:04.675832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:04.675753    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:05.867845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:05.867816    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:05.948160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:05.948128    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:06.045451 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:06.045419    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:06.880727 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:06.880696    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:06.986211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:06.986182    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:07.240064 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.239979    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"]
Sep 10 13:31:07.243536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.243513    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.255863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.255841    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-300680f3-dockercfg-zv7ng\""
Sep 10 13:31:07.263143 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.263126    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-300680f3\""
Sep 10 13:31:07.309037 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.309013    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"]
Sep 10 13:31:07.341383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.341349    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/935a3c40-f10e-4740-bcbc-f6cd45948584-plugins-config-volume\") pod \"payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") " pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.341514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.341405    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz65m\" (UniqueName: \"kubernetes.io/projected/935a3c40-f10e-4740-bcbc-f6cd45948584-kube-api-access-fz65m\") pod \"payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") " pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.442181 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.442157    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/935a3c40-f10e-4740-bcbc-f6cd45948584-plugins-config-volume\") pod \"payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") " pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.442303 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.442185    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz65m\" (UniqueName: \"kubernetes.io/projected/935a3c40-f10e-4740-bcbc-f6cd45948584-kube-api-access-fz65m\") pod \"payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") " pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.442817 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.442798    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/935a3c40-f10e-4740-bcbc-f6cd45948584-plugins-config-volume\") pod \"payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") " pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.459067 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.459040    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz65m\" (UniqueName: \"kubernetes.io/projected/935a3c40-f10e-4740-bcbc-f6cd45948584-kube-api-access-fz65m\") pod \"payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") " pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.556122 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.556095    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:07.865437 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:07.865365    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"]
Sep 10 13:31:07.868701 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:31:07.868649    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod935a3c40_f10e_4740_bcbc_f6cd45948584.slice/crio-1b74a1c82a9a1289e3ddba290dc3274f2d1836f409b50eb180d66466e0c63410 WatchSource:0}: Error finding container 1b74a1c82a9a1289e3ddba290dc3274f2d1836f409b50eb180d66466e0c63410: Status 404 returned error can't find the container with id 1b74a1c82a9a1289e3ddba290dc3274f2d1836f409b50eb180d66466e0c63410
Sep 10 13:31:08.767966 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:08.767931    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv" event={"ID":"935a3c40-f10e-4740-bcbc-f6cd45948584","Type":"ContainerStarted","Data":"b973e2dee73f47f9c4f02b4ccf2933063afb156005b2cb9b62f0f4c5a0961440"}
Sep 10 13:31:08.768436 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:08.767988    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv" event={"ID":"935a3c40-f10e-4740-bcbc-f6cd45948584","Type":"ContainerStarted","Data":"1b74a1c82a9a1289e3ddba290dc3274f2d1836f409b50eb180d66466e0c63410"}
Sep 10 13:31:08.768436 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:08.768057    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:08.805679 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:08.805630    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv" podStartSLOduration=1.8056145959999998 podStartE2EDuration="1.805614596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:08.804685396 +0000 UTC m=+834.015878643" watchObservedRunningTime="2026-09-10 13:31:08.805614596 +0000 UTC m=+834.016807852"
Sep 10 13:31:18.252301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:18.252267    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:18.851400 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:18.851369    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:31:19.774316 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:19.774273    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:22.903386 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:22.903353    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"]
Sep 10 13:31:22.915837 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:22.915809    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:23.681317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:23.681287    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"]
Sep 10 13:31:23.715490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:23.715461    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:23.980823 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:23.980747    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:24.626070 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:24.626041    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"]
Sep 10 13:31:24.650934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:24.650907    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:28.181024 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:28.180992    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:28.277899 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:28.277832    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:34.024760 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:34.024729    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.149658 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.149630    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.213695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.213646    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"]
Sep 10 13:31:35.213913 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.213889    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9" podUID="68361009-7168-4835-a31f-950ed5b8c393" containerName="payload-processing" containerID="cri-o://ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452" gracePeriod=30
Sep 10 13:31:35.230489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.230465    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"]
Sep 10 13:31:35.230721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.230646    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct" podUID="3ed241be-e2bc-41ca-b503-efa8cd259ac7" containerName="payload-pre-processing" containerID="cri-o://41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f" gracePeriod=30
Sep 10 13:31:35.261130 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.261100    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.332410 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.332385    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.492989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.492908    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.497064 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.497042    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:31:35.508063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.508045    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:31:35.560827 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.560798    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.686120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.686059    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs66f\" (UniqueName: \"kubernetes.io/projected/3ed241be-e2bc-41ca-b503-efa8cd259ac7-kube-api-access-fs66f\") pod \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") "
Sep 10 13:31:35.686120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.686088    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ed241be-e2bc-41ca-b503-efa8cd259ac7-plugins-config-volume\") pod \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\" (UID: \"3ed241be-e2bc-41ca-b503-efa8cd259ac7\") "
Sep 10 13:31:35.686320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.686144    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68361009-7168-4835-a31f-950ed5b8c393-plugins-config-volume\") pod \"68361009-7168-4835-a31f-950ed5b8c393\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") "
Sep 10 13:31:35.686320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.686160    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpdgw\" (UniqueName: \"kubernetes.io/projected/68361009-7168-4835-a31f-950ed5b8c393-kube-api-access-cpdgw\") pod \"68361009-7168-4835-a31f-950ed5b8c393\" (UID: \"68361009-7168-4835-a31f-950ed5b8c393\") "
Sep 10 13:31:35.686484 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.686461    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68361009-7168-4835-a31f-950ed5b8c393-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "68361009-7168-4835-a31f-950ed5b8c393" (UID: "68361009-7168-4835-a31f-950ed5b8c393"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:31:35.686560 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.686465    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ed241be-e2bc-41ca-b503-efa8cd259ac7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3ed241be-e2bc-41ca-b503-efa8cd259ac7" (UID: "3ed241be-e2bc-41ca-b503-efa8cd259ac7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:31:35.688194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.688172    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ed241be-e2bc-41ca-b503-efa8cd259ac7-kube-api-access-fs66f" (OuterVolumeSpecName: "kube-api-access-fs66f") pod "3ed241be-e2bc-41ca-b503-efa8cd259ac7" (UID: "3ed241be-e2bc-41ca-b503-efa8cd259ac7"). InnerVolumeSpecName "kube-api-access-fs66f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:35.688270 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.688246    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68361009-7168-4835-a31f-950ed5b8c393-kube-api-access-cpdgw" (OuterVolumeSpecName: "kube-api-access-cpdgw") pod "68361009-7168-4835-a31f-950ed5b8c393" (UID: "68361009-7168-4835-a31f-950ed5b8c393"). InnerVolumeSpecName "kube-api-access-cpdgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:35.787051 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.787026    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs66f\" (UniqueName: \"kubernetes.io/projected/3ed241be-e2bc-41ca-b503-efa8cd259ac7-kube-api-access-fs66f\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.787051 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.787048    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3ed241be-e2bc-41ca-b503-efa8cd259ac7-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.787189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.787059    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68361009-7168-4835-a31f-950ed5b8c393-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.787189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.787067    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpdgw\" (UniqueName: \"kubernetes.io/projected/68361009-7168-4835-a31f-950ed5b8c393-kube-api-access-cpdgw\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:35.841032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.841006    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.902811 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.902788    2551 generic.go:356] "Generic (PLEG): container finished" podID="68361009-7168-4835-a31f-950ed5b8c393" containerID="ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452" exitCode=0
Sep 10 13:31:35.902947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.902830    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"
Sep 10 13:31:35.902947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.902880    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9" event={"ID":"68361009-7168-4835-a31f-950ed5b8c393","Type":"ContainerDied","Data":"ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452"}
Sep 10 13:31:35.902947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.902937    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9" event={"ID":"68361009-7168-4835-a31f-950ed5b8c393","Type":"ContainerDied","Data":"d562b2392bd78d52f1bd761c170c71d978167f79c0eb23ee3f3273df76fd04fe"}
Sep 10 13:31:35.903112 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.902967    2551 scope.go:117] "RemoveContainer" containerID="ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452"
Sep 10 13:31:35.904269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.904245    2551 generic.go:356] "Generic (PLEG): container finished" podID="3ed241be-e2bc-41ca-b503-efa8cd259ac7" containerID="41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f" exitCode=0
Sep 10 13:31:35.904365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.904291    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"
Sep 10 13:31:35.904365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.904299    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct" event={"ID":"3ed241be-e2bc-41ca-b503-efa8cd259ac7","Type":"ContainerDied","Data":"41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f"}
Sep 10 13:31:35.904365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.904337    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct" event={"ID":"3ed241be-e2bc-41ca-b503-efa8cd259ac7","Type":"ContainerDied","Data":"9afc47756737c473d21dd81193963724d3a4c7a970708a0e68cfabcb97c4f3f5"}
Sep 10 13:31:35.912446 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.912279    2551 scope.go:117] "RemoveContainer" containerID="ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452"
Sep 10 13:31:35.912678 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:31:35.912643    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452\": container with ID starting with ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452 not found: ID does not exist" containerID="ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452"
Sep 10 13:31:35.912795 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.912713    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452"} err="failed to get container status \"ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452\": rpc error: code = NotFound desc = could not find container \"ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452\": container with ID starting with ed32ac73cf4fdd454feea02d0a1167a542e68d3f6bc94ecfa94efad05590e452 not found: ID does not exist"
Sep 10 13:31:35.912795 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.912739    2551 scope.go:117] "RemoveContainer" containerID="41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f"
Sep 10 13:31:35.920945 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.920930    2551 scope.go:117] "RemoveContainer" containerID="41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f"
Sep 10 13:31:35.921200 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:31:35.921181    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f\": container with ID starting with 41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f not found: ID does not exist" containerID="41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f"
Sep 10 13:31:35.921257 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.921206    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f"} err="failed to get container status \"41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f\": rpc error: code = NotFound desc = could not find container \"41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f\": container with ID starting with 41c6cb3b9daa0a4957fd9d9fe2057ec299d09a1cbd356fc96aad3a325318639f not found: ID does not exist"
Sep 10 13:31:35.931300 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.931282    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"]
Sep 10 13:31:35.934542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.934518    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:35.935989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.935972    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-af59ca-97fdc9599-g9gj9"]
Sep 10 13:31:35.948675 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.948612    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"]
Sep 10 13:31:35.954274 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:35.954250    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-af59ca-bc75c85d6-szbct"]
Sep 10 13:31:36.728783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.728752    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"]
Sep 10 13:31:36.729266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.729007    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv" podUID="935a3c40-f10e-4740-bcbc-f6cd45948584" containerName="payload-processing" containerID="cri-o://b973e2dee73f47f9c4f02b4ccf2933063afb156005b2cb9b62f0f4c5a0961440" gracePeriod=30
Sep 10 13:31:36.733539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.733503    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:36.835825 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.835797    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:36.912003 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.911967    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:36.913108 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.913083    2551 generic.go:356] "Generic (PLEG): container finished" podID="935a3c40-f10e-4740-bcbc-f6cd45948584" containerID="b973e2dee73f47f9c4f02b4ccf2933063afb156005b2cb9b62f0f4c5a0961440" exitCode=0
Sep 10 13:31:36.913214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.913144    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv" event={"ID":"935a3c40-f10e-4740-bcbc-f6cd45948584","Type":"ContainerDied","Data":"b973e2dee73f47f9c4f02b4ccf2933063afb156005b2cb9b62f0f4c5a0961440"}
Sep 10 13:31:36.972804 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.972775    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:36.989654 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:36.989603    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:37.099147 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.099116    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/935a3c40-f10e-4740-bcbc-f6cd45948584-plugins-config-volume\") pod \"935a3c40-f10e-4740-bcbc-f6cd45948584\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") "
Sep 10 13:31:37.099281 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.099237    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz65m\" (UniqueName: \"kubernetes.io/projected/935a3c40-f10e-4740-bcbc-f6cd45948584-kube-api-access-fz65m\") pod \"935a3c40-f10e-4740-bcbc-f6cd45948584\" (UID: \"935a3c40-f10e-4740-bcbc-f6cd45948584\") "
Sep 10 13:31:37.099422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.099402    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/935a3c40-f10e-4740-bcbc-f6cd45948584-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "935a3c40-f10e-4740-bcbc-f6cd45948584" (UID: "935a3c40-f10e-4740-bcbc-f6cd45948584"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:31:37.099513 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.099500    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/935a3c40-f10e-4740-bcbc-f6cd45948584-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:37.101221 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.101190    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/935a3c40-f10e-4740-bcbc-f6cd45948584-kube-api-access-fz65m" (OuterVolumeSpecName: "kube-api-access-fz65m") pod "935a3c40-f10e-4740-bcbc-f6cd45948584" (UID: "935a3c40-f10e-4740-bcbc-f6cd45948584"). InnerVolumeSpecName "kube-api-access-fz65m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:37.200368 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.200339    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz65m\" (UniqueName: \"kubernetes.io/projected/935a3c40-f10e-4740-bcbc-f6cd45948584-kube-api-access-fz65m\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:37.248081 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.248016    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ed241be-e2bc-41ca-b503-efa8cd259ac7" path="/var/lib/kubelet/pods/3ed241be-e2bc-41ca-b503-efa8cd259ac7/volumes"
Sep 10 13:31:37.248362 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.248350    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68361009-7168-4835-a31f-950ed5b8c393" path="/var/lib/kubelet/pods/68361009-7168-4835-a31f-950ed5b8c393/volumes"
Sep 10 13:31:37.919604 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.919576    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"
Sep 10 13:31:37.920034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.919573    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv" event={"ID":"935a3c40-f10e-4740-bcbc-f6cd45948584","Type":"ContainerDied","Data":"1b74a1c82a9a1289e3ddba290dc3274f2d1836f409b50eb180d66466e0c63410"}
Sep 10 13:31:37.920034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.919727    2551 scope.go:117] "RemoveContainer" containerID="b973e2dee73f47f9c4f02b4ccf2933063afb156005b2cb9b62f0f4c5a0961440"
Sep 10 13:31:37.922687 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.922627    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h" podUID="8234dda6-0e9c-47f1-93a0-12ebb265580e" containerName="main" containerID="cri-o://38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5" gracePeriod=45
Sep 10 13:31:37.942224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.942201    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"]
Sep 10 13:31:37.949799 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:37.949776    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-300680f3-59c5d8b44c-qpmqv"]
Sep 10 13:31:38.137941 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.137913    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:38.181638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.181585    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:31:38.207048 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207023    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww274\" (UniqueName: \"kubernetes.io/projected/8234dda6-0e9c-47f1-93a0-12ebb265580e-kube-api-access-ww274\") pod \"8234dda6-0e9c-47f1-93a0-12ebb265580e\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") "
Sep 10 13:31:38.207211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207062    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-model-cache\") pod \"8234dda6-0e9c-47f1-93a0-12ebb265580e\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") "
Sep 10 13:31:38.207211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207088    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-home\") pod \"8234dda6-0e9c-47f1-93a0-12ebb265580e\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") "
Sep 10 13:31:38.207211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207134    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-tmp-dir\") pod \"8234dda6-0e9c-47f1-93a0-12ebb265580e\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") "
Sep 10 13:31:38.207211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207156    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-dshm\") pod \"8234dda6-0e9c-47f1-93a0-12ebb265580e\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") "
Sep 10 13:31:38.207451 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207213    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8234dda6-0e9c-47f1-93a0-12ebb265580e-tls-certs\") pod \"8234dda6-0e9c-47f1-93a0-12ebb265580e\" (UID: \"8234dda6-0e9c-47f1-93a0-12ebb265580e\") "
Sep 10 13:31:38.207451 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207332    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-model-cache" (OuterVolumeSpecName: "model-cache") pod "8234dda6-0e9c-47f1-93a0-12ebb265580e" (UID: "8234dda6-0e9c-47f1-93a0-12ebb265580e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:38.207451 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207408    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8234dda6-0e9c-47f1-93a0-12ebb265580e" (UID: "8234dda6-0e9c-47f1-93a0-12ebb265580e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:38.207619 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207552    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:38.207696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207621    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:38.207696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.207629    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-home" (OuterVolumeSpecName: "home") pod "8234dda6-0e9c-47f1-93a0-12ebb265580e" (UID: "8234dda6-0e9c-47f1-93a0-12ebb265580e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:38.209655 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.209629    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8234dda6-0e9c-47f1-93a0-12ebb265580e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8234dda6-0e9c-47f1-93a0-12ebb265580e" (UID: "8234dda6-0e9c-47f1-93a0-12ebb265580e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:31:38.209791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.209745    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-dshm" (OuterVolumeSpecName: "dshm") pod "8234dda6-0e9c-47f1-93a0-12ebb265580e" (UID: "8234dda6-0e9c-47f1-93a0-12ebb265580e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:38.209791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.209770    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8234dda6-0e9c-47f1-93a0-12ebb265580e-kube-api-access-ww274" (OuterVolumeSpecName: "kube-api-access-ww274") pod "8234dda6-0e9c-47f1-93a0-12ebb265580e" (UID: "8234dda6-0e9c-47f1-93a0-12ebb265580e"). InnerVolumeSpecName "kube-api-access-ww274". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:38.308000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.307962    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww274\" (UniqueName: \"kubernetes.io/projected/8234dda6-0e9c-47f1-93a0-12ebb265580e-kube-api-access-ww274\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:38.308000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.307997    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:38.308000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.308010    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8234dda6-0e9c-47f1-93a0-12ebb265580e-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:38.308222 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.308021    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8234dda6-0e9c-47f1-93a0-12ebb265580e-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:38.515081 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.515002    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:31:38.594102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.594071    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:38.700268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.700231    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr" podUID="f072cce5-1269-4450-9e55-3ae092f1a740" containerName="main" containerID="cri-o://dbb541190c9a0ee2df9da54e7fe98d16cceef658569092689fe2dc8e816e56eb" gracePeriod=45
Sep 10 13:31:38.928537 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.928507    2551 generic.go:356] "Generic (PLEG): container finished" podID="8234dda6-0e9c-47f1-93a0-12ebb265580e" containerID="38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5" exitCode=0
Sep 10 13:31:38.929320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.928552    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"
Sep 10 13:31:38.929320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.928592    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h" event={"ID":"8234dda6-0e9c-47f1-93a0-12ebb265580e","Type":"ContainerDied","Data":"38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5"}
Sep 10 13:31:38.929320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.928637    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h" event={"ID":"8234dda6-0e9c-47f1-93a0-12ebb265580e","Type":"ContainerDied","Data":"662c79ef325862a32563949bdc4cabed97101f4c7e2a11ef05048ad683efa008"}
Sep 10 13:31:38.929320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.928657    2551 scope.go:117] "RemoveContainer" containerID="38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5"
Sep 10 13:31:38.930374 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.930353    2551 generic.go:356] "Generic (PLEG): container finished" podID="f072cce5-1269-4450-9e55-3ae092f1a740" containerID="dbb541190c9a0ee2df9da54e7fe98d16cceef658569092689fe2dc8e816e56eb" exitCode=0
Sep 10 13:31:38.930495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.930428    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr" event={"ID":"f072cce5-1269-4450-9e55-3ae092f1a740","Type":"ContainerDied","Data":"dbb541190c9a0ee2df9da54e7fe98d16cceef658569092689fe2dc8e816e56eb"}
Sep 10 13:31:38.935165 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.935145    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:38.935463 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.935445    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:31:38.947396 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.947379    2551 scope.go:117] "RemoveContainer" containerID="38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5"
Sep 10 13:31:38.947711 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:31:38.947684    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5\": container with ID starting with 38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5 not found: ID does not exist" containerID="38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5"
Sep 10 13:31:38.947801 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.947724    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5"} err="failed to get container status \"38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5\": rpc error: code = NotFound desc = could not find container \"38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5\": container with ID starting with 38db037bb8f6edc20b4513c13831d0b5d223e33d0f1935a374a169157b4c17c5 not found: ID does not exist"
Sep 10 13:31:38.999055 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:38.999026    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"]
Sep 10 13:31:39.009236 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.009212    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-5995df69-kserve-7b475d986b-zhb7h"]
Sep 10 13:31:39.013839 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.013823    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-tmp-dir\") pod \"f072cce5-1269-4450-9e55-3ae092f1a740\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") "
Sep 10 13:31:39.013904 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.013853    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-dshm\") pod \"f072cce5-1269-4450-9e55-3ae092f1a740\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") "
Sep 10 13:31:39.013904 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.013877    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f072cce5-1269-4450-9e55-3ae092f1a740-tls-certs\") pod \"f072cce5-1269-4450-9e55-3ae092f1a740\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") "
Sep 10 13:31:39.014000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.013931    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-model-cache\") pod \"f072cce5-1269-4450-9e55-3ae092f1a740\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") "
Sep 10 13:31:39.014000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.013967    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-home\") pod \"f072cce5-1269-4450-9e55-3ae092f1a740\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") "
Sep 10 13:31:39.014000 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.013989    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2d4h\" (UniqueName: \"kubernetes.io/projected/f072cce5-1269-4450-9e55-3ae092f1a740-kube-api-access-w2d4h\") pod \"f072cce5-1269-4450-9e55-3ae092f1a740\" (UID: \"f072cce5-1269-4450-9e55-3ae092f1a740\") "
Sep 10 13:31:39.014224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.014136    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f072cce5-1269-4450-9e55-3ae092f1a740" (UID: "f072cce5-1269-4450-9e55-3ae092f1a740"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.014224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.014193    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-model-cache" (OuterVolumeSpecName: "model-cache") pod "f072cce5-1269-4450-9e55-3ae092f1a740" (UID: "f072cce5-1269-4450-9e55-3ae092f1a740"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.014224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.014217    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-home" (OuterVolumeSpecName: "home") pod "f072cce5-1269-4450-9e55-3ae092f1a740" (UID: "f072cce5-1269-4450-9e55-3ae092f1a740"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.014439 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.014265    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:39.016138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.016113    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f072cce5-1269-4450-9e55-3ae092f1a740-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f072cce5-1269-4450-9e55-3ae092f1a740" (UID: "f072cce5-1269-4450-9e55-3ae092f1a740"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:31:39.016138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.016129    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f072cce5-1269-4450-9e55-3ae092f1a740-kube-api-access-w2d4h" (OuterVolumeSpecName: "kube-api-access-w2d4h") pod "f072cce5-1269-4450-9e55-3ae092f1a740" (UID: "f072cce5-1269-4450-9e55-3ae092f1a740"). InnerVolumeSpecName "kube-api-access-w2d4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:39.016261 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.016183    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-dshm" (OuterVolumeSpecName: "dshm") pod "f072cce5-1269-4450-9e55-3ae092f1a740" (UID: "f072cce5-1269-4450-9e55-3ae092f1a740"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.115537 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.115457    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:39.115537 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.115485    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:39.115537 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.115494    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2d4h\" (UniqueName: \"kubernetes.io/projected/f072cce5-1269-4450-9e55-3ae092f1a740-kube-api-access-w2d4h\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:39.115537 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.115505    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f072cce5-1269-4450-9e55-3ae092f1a740-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:39.115537 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.115513    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f072cce5-1269-4450-9e55-3ae092f1a740-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:39.232854 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.232798    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:39.249118 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.249091    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8234dda6-0e9c-47f1-93a0-12ebb265580e" path="/var/lib/kubelet/pods/8234dda6-0e9c-47f1-93a0-12ebb265580e/volumes"
Sep 10 13:31:39.249479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.249460    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="935a3c40-f10e-4740-bcbc-f6cd45948584" path="/var/lib/kubelet/pods/935a3c40-f10e-4740-bcbc-f6cd45948584/volumes"
Sep 10 13:31:39.645511 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.645451    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr" podUID="c2ed8d28-804b-4824-8440-d627a2b6d24c" containerName="main" containerID="cri-o://8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b" gracePeriod=45
Sep 10 13:31:39.885017 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.884996    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:31:39.921769 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.921709    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmq6m\" (UniqueName: \"kubernetes.io/projected/c2ed8d28-804b-4824-8440-d627a2b6d24c-kube-api-access-hmq6m\") pod \"c2ed8d28-804b-4824-8440-d627a2b6d24c\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") "
Sep 10 13:31:39.921769 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.921744    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-model-cache\") pod \"c2ed8d28-804b-4824-8440-d627a2b6d24c\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") "
Sep 10 13:31:39.921769 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.921769    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2ed8d28-804b-4824-8440-d627a2b6d24c-tls-certs\") pod \"c2ed8d28-804b-4824-8440-d627a2b6d24c\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") "
Sep 10 13:31:39.921963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.921829    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-dshm\") pod \"c2ed8d28-804b-4824-8440-d627a2b6d24c\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") "
Sep 10 13:31:39.921963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.921890    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-tmp-dir\") pod \"c2ed8d28-804b-4824-8440-d627a2b6d24c\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") "
Sep 10 13:31:39.921963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.921914    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-home\") pod \"c2ed8d28-804b-4824-8440-d627a2b6d24c\" (UID: \"c2ed8d28-804b-4824-8440-d627a2b6d24c\") "
Sep 10 13:31:39.922074 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.922018    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2ed8d28-804b-4824-8440-d627a2b6d24c" (UID: "c2ed8d28-804b-4824-8440-d627a2b6d24c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.922213 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.922189    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:39.922213 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.922210    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-home" (OuterVolumeSpecName: "home") pod "c2ed8d28-804b-4824-8440-d627a2b6d24c" (UID: "c2ed8d28-804b-4824-8440-d627a2b6d24c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.922400 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.922324    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2ed8d28-804b-4824-8440-d627a2b6d24c" (UID: "c2ed8d28-804b-4824-8440-d627a2b6d24c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.923921 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.923899    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-dshm" (OuterVolumeSpecName: "dshm") pod "c2ed8d28-804b-4824-8440-d627a2b6d24c" (UID: "c2ed8d28-804b-4824-8440-d627a2b6d24c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:31:39.923996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.923954    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2ed8d28-804b-4824-8440-d627a2b6d24c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2ed8d28-804b-4824-8440-d627a2b6d24c" (UID: "c2ed8d28-804b-4824-8440-d627a2b6d24c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:31:39.924219 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.924203    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2ed8d28-804b-4824-8440-d627a2b6d24c-kube-api-access-hmq6m" (OuterVolumeSpecName: "kube-api-access-hmq6m") pod "c2ed8d28-804b-4824-8440-d627a2b6d24c" (UID: "c2ed8d28-804b-4824-8440-d627a2b6d24c"). InnerVolumeSpecName "kube-api-access-hmq6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:31:39.938011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.937986    2551 generic.go:356] "Generic (PLEG): container finished" podID="c2ed8d28-804b-4824-8440-d627a2b6d24c" containerID="8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b" exitCode=0
Sep 10 13:31:39.938371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.938021    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr" event={"ID":"c2ed8d28-804b-4824-8440-d627a2b6d24c","Type":"ContainerDied","Data":"8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b"}
Sep 10 13:31:39.938371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.938038    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"
Sep 10 13:31:39.938371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.938075    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr" event={"ID":"c2ed8d28-804b-4824-8440-d627a2b6d24c","Type":"ContainerDied","Data":"b1262d860eeee4cdd84cba6f2b7b16e42e289a45b523454bd79791fd9182f863"}
Sep 10 13:31:39.938371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.938099    2551 scope.go:117] "RemoveContainer" containerID="8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b"
Sep 10 13:31:39.939877 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.939815    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"
Sep 10 13:31:39.939877 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.939857    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr" event={"ID":"f072cce5-1269-4450-9e55-3ae092f1a740","Type":"ContainerDied","Data":"f60185d4e37e9b3464aeeba7ee8ed1dc9c8a3a395bfea62810742967b2aa21a8"}
Sep 10 13:31:39.955202 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.955182    2551 scope.go:117] "RemoveContainer" containerID="8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b"
Sep 10 13:31:39.955524 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:31:39.955492    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b\": container with ID starting with 8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b not found: ID does not exist" containerID="8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b"
Sep 10 13:31:39.955586 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.955535    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b"} err="failed to get container status \"8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b\": rpc error: code = NotFound desc = could not find container \"8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b\": container with ID starting with 8651de16f138d447780d0c653f7820d0a76bb5ec9aeda63acbd56f43a295901b not found: ID does not exist"
Sep 10 13:31:39.955586 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.955557    2551 scope.go:117] "RemoveContainer" containerID="dbb541190c9a0ee2df9da54e7fe98d16cceef658569092689fe2dc8e816e56eb"
Sep 10 13:31:39.971053 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.971028    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"]
Sep 10 13:31:39.975494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.975473    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-5995df69-kserve-5bc76888bd-vhdpr"]
Sep 10 13:31:39.989797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.989777    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"]
Sep 10 13:31:39.994996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:39.994974    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-af59ca/autores-model-af59ca-kserve-5885dd79c5-2qxpr"]
Sep 10 13:31:40.022994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:40.022972    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmq6m\" (UniqueName: \"kubernetes.io/projected/c2ed8d28-804b-4824-8440-d627a2b6d24c-kube-api-access-hmq6m\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:40.022994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:40.022995    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2ed8d28-804b-4824-8440-d627a2b6d24c-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:40.023116 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:40.023006    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:40.023116 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:40.023034    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:40.023116 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:40.023042    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2ed8d28-804b-4824-8440-d627a2b6d24c-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:31:41.248367 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:41.248338    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2ed8d28-804b-4824-8440-d627a2b6d24c" path="/var/lib/kubelet/pods/c2ed8d28-804b-4824-8440-d627a2b6d24c/volumes"
Sep 10 13:31:41.248755 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:41.248732    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f072cce5-1269-4450-9e55-3ae092f1a740" path="/var/lib/kubelet/pods/f072cce5-1269-4450-9e55-3ae092f1a740/volumes"
Sep 10 13:31:41.769402 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:41.769375    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:41.864862 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:41.864829    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:43.012303 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.012275    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:43.107928 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.107895    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:43.290167 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.290131    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:43.390427 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.390372    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:43.742388 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.742298    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:43.855468 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.855412    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:43.924229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924196    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"]
Sep 10 13:31:43.924545 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924532    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ed241be-e2bc-41ca-b503-efa8cd259ac7" containerName="payload-pre-processing"
Sep 10 13:31:43.924589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924549    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ed241be-e2bc-41ca-b503-efa8cd259ac7" containerName="payload-pre-processing"
Sep 10 13:31:43.924589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924560    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8234dda6-0e9c-47f1-93a0-12ebb265580e" containerName="main"
Sep 10 13:31:43.924589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924567    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8234dda6-0e9c-47f1-93a0-12ebb265580e" containerName="main"
Sep 10 13:31:43.924589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924580    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2ed8d28-804b-4824-8440-d627a2b6d24c" containerName="main"
Sep 10 13:31:43.924589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924587    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2ed8d28-804b-4824-8440-d627a2b6d24c" containerName="main"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924594    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="935a3c40-f10e-4740-bcbc-f6cd45948584" containerName="payload-processing"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924599    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="935a3c40-f10e-4740-bcbc-f6cd45948584" containerName="payload-processing"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924611    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68361009-7168-4835-a31f-950ed5b8c393" containerName="payload-processing"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924616    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="68361009-7168-4835-a31f-950ed5b8c393" containerName="payload-processing"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924627    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f072cce5-1269-4450-9e55-3ae092f1a740" containerName="main"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924635    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f072cce5-1269-4450-9e55-3ae092f1a740" containerName="main"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924705    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ed241be-e2bc-41ca-b503-efa8cd259ac7" containerName="payload-pre-processing"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924718    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="935a3c40-f10e-4740-bcbc-f6cd45948584" containerName="payload-processing"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924726    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8234dda6-0e9c-47f1-93a0-12ebb265580e" containerName="main"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924732    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f072cce5-1269-4450-9e55-3ae092f1a740" containerName="main"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924737    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="68361009-7168-4835-a31f-950ed5b8c393" containerName="payload-processing"
Sep 10 13:31:43.924766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.924744    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2ed8d28-804b-4824-8440-d627a2b6d24c" containerName="main"
Sep 10 13:31:43.927711 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.927696    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:43.931511 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.931491    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-d46083cd-dockercfg-97kkv\""
Sep 10 13:31:43.932879 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.932861    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-d46083cd\""
Sep 10 13:31:43.944463 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.944442    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"]
Sep 10 13:31:43.956735 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.956712    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27ccr\" (UniqueName: \"kubernetes.io/projected/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-kube-api-access-27ccr\") pod \"payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:43.956842 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:43.956761    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:44.030703 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.030655    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"]
Sep 10 13:31:44.033973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.033958    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.058183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.058150    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27ccr\" (UniqueName: \"kubernetes.io/projected/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-kube-api-access-27ccr\") pod \"payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:44.058365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.058205    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e8eda21-6a5c-4543-ab8c-c67e87864524-plugins-config-volume\") pod \"payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") " pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.058365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.058254    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vljql\" (UniqueName: \"kubernetes.io/projected/9e8eda21-6a5c-4543-ab8c-c67e87864524-kube-api-access-vljql\") pod \"payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") " pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.058365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.058289    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:44.059211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.059178    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:44.060923 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.060900    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"]
Sep 10 13:31:44.073714 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.073681    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27ccr\" (UniqueName: \"kubernetes.io/projected/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-kube-api-access-27ccr\") pod \"payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:44.159632 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.159597    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e8eda21-6a5c-4543-ab8c-c67e87864524-plugins-config-volume\") pod \"payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") " pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.159809 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.159654    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljql\" (UniqueName: \"kubernetes.io/projected/9e8eda21-6a5c-4543-ab8c-c67e87864524-kube-api-access-vljql\") pod \"payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") " pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.160265 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.160246    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e8eda21-6a5c-4543-ab8c-c67e87864524-plugins-config-volume\") pod \"payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") " pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.180724 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.180694    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljql\" (UniqueName: \"kubernetes.io/projected/9e8eda21-6a5c-4543-ab8c-c67e87864524-kube-api-access-vljql\") pod \"payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") " pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.237542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.237509    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:44.344037 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.344004    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.534918 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.534894    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"]
Sep 10 13:31:44.536500 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:31:44.536476    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a184cf8_7ff2_4fa4_9cd8_5b80240fde36.slice/crio-90e4c970530e77c12fbf7e49f1465c2e398341fab4649e436fa67ef45990b66a WatchSource:0}: Error finding container 90e4c970530e77c12fbf7e49f1465c2e398341fab4649e436fa67ef45990b66a: Status 404 returned error can't find the container with id 90e4c970530e77c12fbf7e49f1465c2e398341fab4649e436fa67ef45990b66a
Sep 10 13:31:44.649805 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.649773    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"]
Sep 10 13:31:44.652953 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:31:44.652926    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8eda21_6a5c_4543_ab8c_c67e87864524.slice/crio-ec3a415483b62bb947593f0a48572070f89c16e9e27573700edc4dba1d2e7719 WatchSource:0}: Error finding container ec3a415483b62bb947593f0a48572070f89c16e9e27573700edc4dba1d2e7719: Status 404 returned error can't find the container with id ec3a415483b62bb947593f0a48572070f89c16e9e27573700edc4dba1d2e7719
Sep 10 13:31:44.970138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.970043    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87" event={"ID":"9e8eda21-6a5c-4543-ab8c-c67e87864524","Type":"ContainerStarted","Data":"df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858"}
Sep 10 13:31:44.970301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.970138    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87" event={"ID":"9e8eda21-6a5c-4543-ab8c-c67e87864524","Type":"ContainerStarted","Data":"ec3a415483b62bb947593f0a48572070f89c16e9e27573700edc4dba1d2e7719"}
Sep 10 13:31:44.970301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.970165    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:44.971703 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.971657    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86" event={"ID":"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36","Type":"ContainerStarted","Data":"807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021"}
Sep 10 13:31:44.971837 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.971718    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86" event={"ID":"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36","Type":"ContainerStarted","Data":"90e4c970530e77c12fbf7e49f1465c2e398341fab4649e436fa67ef45990b66a"}
Sep 10 13:31:44.971837 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:44.971757    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:31:45.024306 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:45.024261    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87" podStartSLOduration=2.024246037 podStartE2EDuration="2.024246037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:45.022451697 +0000 UTC m=+870.233644966" watchObservedRunningTime="2026-09-10 13:31:45.024246037 +0000 UTC m=+870.235439293"
Sep 10 13:31:45.064305 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:45.064262    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86" podStartSLOduration=2.064247034 podStartE2EDuration="2.064247034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:45.062973549 +0000 UTC m=+870.274166808" watchObservedRunningTime="2026-09-10 13:31:45.064247034 +0000 UTC m=+870.275440291"
Sep 10 13:31:45.216282 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:45.216245    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:45.399928 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:45.399880    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:45.780414 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:45.780383    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:46.040796 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:46.040717    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:46.248648 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:46.248615    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:46.733774 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:46.733740    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:47.520828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:47.520791    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:48.006846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:48.006748    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:50.345251 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:50.345211    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:50.448024 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:50.447994    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:51.558740 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:51.558711    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:51.694720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:51.694693    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:52.127274 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.127242    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:52.622964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.622932    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"]
Sep 10 13:31:52.627241 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.627218    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"]
Sep 10 13:31:52.627376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.627331    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.630403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.630385    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:52.631731 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.631716    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-a26db0-dockercfg-pzsdm\""
Sep 10 13:31:52.631803 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.631717    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-a26db0\""
Sep 10 13:31:52.639103 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.639079    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"]
Sep 10 13:31:52.643245 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.643227    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"]
Sep 10 13:31:52.737365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.737336    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74z77\" (UniqueName: \"kubernetes.io/projected/aaf0b995-d33a-469e-98c5-605baf08e80f-kube-api-access-74z77\") pod \"payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") " pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:52.737499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.737385    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64p9m\" (UniqueName: \"kubernetes.io/projected/0b62ede4-a34f-43a9-b11a-883efa72889f-kube-api-access-64p9m\") pod \"payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.737499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.737406    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b62ede4-a34f-43a9-b11a-883efa72889f-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.737499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.737485    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaf0b995-d33a-469e-98c5-605baf08e80f-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") " pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:52.838059 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.838028    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74z77\" (UniqueName: \"kubernetes.io/projected/aaf0b995-d33a-469e-98c5-605baf08e80f-kube-api-access-74z77\") pod \"payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") " pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:52.838194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.838080    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64p9m\" (UniqueName: \"kubernetes.io/projected/0b62ede4-a34f-43a9-b11a-883efa72889f-kube-api-access-64p9m\") pod \"payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.838194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.838120    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b62ede4-a34f-43a9-b11a-883efa72889f-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.838194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.838175    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaf0b995-d33a-469e-98c5-605baf08e80f-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") " pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:52.838926 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.838908    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b62ede4-a34f-43a9-b11a-883efa72889f-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.838972 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.838915    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaf0b995-d33a-469e-98c5-605baf08e80f-plugins-config-volume\") pod \"payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") " pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:52.848201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.848180    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74z77\" (UniqueName: \"kubernetes.io/projected/aaf0b995-d33a-469e-98c5-605baf08e80f-kube-api-access-74z77\") pod \"payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") " pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:52.848328 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.848272    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64p9m\" (UniqueName: \"kubernetes.io/projected/0b62ede4-a34f-43a9-b11a-883efa72889f-kube-api-access-64p9m\") pod \"payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.941002 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.940943    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:52.949049 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:52.949033    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:53.230819 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:53.230795    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"]
Sep 10 13:31:53.232416 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:31:53.232387    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b62ede4_a34f_43a9_b11a_883efa72889f.slice/crio-1d9063324d87982f77871107af0d49bdaad5dc79405e62c2814a0c1542d893f1 WatchSource:0}: Error finding container 1d9063324d87982f77871107af0d49bdaad5dc79405e62c2814a0c1542d893f1: Status 404 returned error can't find the container with id 1d9063324d87982f77871107af0d49bdaad5dc79405e62c2814a0c1542d893f1
Sep 10 13:31:53.248608 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:31:53.248522    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf0b995_d33a_469e_98c5_605baf08e80f.slice/crio-eaf59ed5e0f1957c530d55e3ea06ba61c689d5a5c848153735ac5b258bab21c1 WatchSource:0}: Error finding container eaf59ed5e0f1957c530d55e3ea06ba61c689d5a5c848153735ac5b258bab21c1: Status 404 returned error can't find the container with id eaf59ed5e0f1957c530d55e3ea06ba61c689d5a5c848153735ac5b258bab21c1
Sep 10 13:31:53.250177 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:53.250106    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"]
Sep 10 13:31:53.448029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:53.447986    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:53.550241 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:53.550211    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:31:54.021953 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.021913    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c" event={"ID":"0b62ede4-a34f-43a9-b11a-883efa72889f","Type":"ContainerStarted","Data":"44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce"}
Sep 10 13:31:54.022397 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.021984    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c" event={"ID":"0b62ede4-a34f-43a9-b11a-883efa72889f","Type":"ContainerStarted","Data":"1d9063324d87982f77871107af0d49bdaad5dc79405e62c2814a0c1542d893f1"}
Sep 10 13:31:54.022397 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.022011    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:31:54.023354 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.023332    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g" event={"ID":"aaf0b995-d33a-469e-98c5-605baf08e80f","Type":"ContainerStarted","Data":"a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17"}
Sep 10 13:31:54.023479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.023375    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g" event={"ID":"aaf0b995-d33a-469e-98c5-605baf08e80f","Type":"ContainerStarted","Data":"eaf59ed5e0f1957c530d55e3ea06ba61c689d5a5c848153735ac5b258bab21c1"}
Sep 10 13:31:54.023479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.023443    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:31:54.054263 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.054201    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c" podStartSLOduration=2.054182631 podStartE2EDuration="2.054182631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:54.052547093 +0000 UTC m=+879.263740349" watchObservedRunningTime="2026-09-10 13:31:54.054182631 +0000 UTC m=+879.265375892"
Sep 10 13:31:54.081766 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:54.081725    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g" podStartSLOduration=2.081710479 podStartE2EDuration="2.081710479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:31:54.08058647 +0000 UTC m=+879.291779730" watchObservedRunningTime="2026-09-10 13:31:54.081710479 +0000 UTC m=+879.292903736"
Sep 10 13:31:55.980149 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:55.980119    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:31:55.980532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:31:55.980402    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:32:00.805037 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:00.805001    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:32:05.030173 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:05.030142    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:32:05.030716 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:05.030212    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:32:09.297919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:09.297840    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:11.758628 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:11.758590    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"]
Sep 10 13:32:11.763128 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:11.763112    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"
Sep 10 13:32:11.766084 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:11.766068    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:32:11.766291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:11.766279    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:32:11.779166 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:11.779138    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"]
Sep 10 13:32:11.910122 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:11.910083    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4561679e-0da6-4d92-8a23-6c90a5d21dcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d46083cd-2khn6\" (UID: \"4561679e-0da6-4d92-8a23-6c90a5d21dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"
Sep 10 13:32:12.011077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:12.010984    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4561679e-0da6-4d92-8a23-6c90a5d21dcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d46083cd-2khn6\" (UID: \"4561679e-0da6-4d92-8a23-6c90a5d21dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"
Sep 10 13:32:12.011570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:12.011549    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4561679e-0da6-4d92-8a23-6c90a5d21dcc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-d46083cd-2khn6\" (UID: \"4561679e-0da6-4d92-8a23-6c90a5d21dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"
Sep 10 13:32:12.073275 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:12.073241    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"
Sep 10 13:32:12.346550 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:12.346433    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"]
Sep 10 13:32:12.348731 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:32:12.348708    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice/crio-77a90b4d2bda5b6869840c28f98beedcf4a42cc95110aa22230c0241c991aa52 WatchSource:0}: Error finding container 77a90b4d2bda5b6869840c28f98beedcf4a42cc95110aa22230c0241c991aa52: Status 404 returned error can't find the container with id 77a90b4d2bda5b6869840c28f98beedcf4a42cc95110aa22230c0241c991aa52
Sep 10 13:32:13.122368 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:13.122333    2551 generic.go:356] "Generic (PLEG): container finished" podID="4561679e-0da6-4d92-8a23-6c90a5d21dcc" containerID="8e5badfbf3cb08be2cc4766c10c0482f1cccff10e448769bc8a0354af5167bd9" exitCode=0
Sep 10 13:32:13.122736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:13.122412    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6" event={"ID":"4561679e-0da6-4d92-8a23-6c90a5d21dcc","Type":"ContainerDied","Data":"8e5badfbf3cb08be2cc4766c10c0482f1cccff10e448769bc8a0354af5167bd9"}
Sep 10 13:32:13.122736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:13.122468    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6" event={"ID":"4561679e-0da6-4d92-8a23-6c90a5d21dcc","Type":"ContainerStarted","Data":"77a90b4d2bda5b6869840c28f98beedcf4a42cc95110aa22230c0241c991aa52"}
Sep 10 13:32:14.262848 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:14.262821    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"
Sep 10 13:32:14.434571 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:14.434491    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4561679e-0da6-4d92-8a23-6c90a5d21dcc-maas-api-service-ca\") pod \"4561679e-0da6-4d92-8a23-6c90a5d21dcc\" (UID: \"4561679e-0da6-4d92-8a23-6c90a5d21dcc\") "
Sep 10 13:32:14.434833 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:14.434812    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4561679e-0da6-4d92-8a23-6c90a5d21dcc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4561679e-0da6-4d92-8a23-6c90a5d21dcc" (UID: "4561679e-0da6-4d92-8a23-6c90a5d21dcc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:32:14.536152 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:14.536121    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4561679e-0da6-4d92-8a23-6c90a5d21dcc-maas-api-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:32:14.781201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:14.781172    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:15.133307 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:15.133229    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"
Sep 10 13:32:15.133307 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:15.133260    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6" event={"ID":"4561679e-0da6-4d92-8a23-6c90a5d21dcc","Type":"ContainerDied","Data":"77a90b4d2bda5b6869840c28f98beedcf4a42cc95110aa22230c0241c991aa52"}
Sep 10 13:32:15.133508 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:15.133341    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="77a90b4d2bda5b6869840c28f98beedcf4a42cc95110aa22230c0241c991aa52"
Sep 10 13:32:15.186522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:15.186498    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:32:15.186812 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:15.186795    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:32:15.704903 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:15.704872    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:32:17.406259 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:17.406225    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:20.509390 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:20.509334    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:23.380503 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:23.380473    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:24.476249 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.476215    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:24.485189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.485163    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"]
Sep 10 13:32:24.485410 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.485367    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87" podUID="9e8eda21-6a5c-4543-ab8c-c67e87864524" containerName="payload-processing" containerID="cri-o://df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858" gracePeriod=30
Sep 10 13:32:24.499803 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.499778    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"]
Sep 10 13:32:24.500013 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.499990    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86" podUID="8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" containerName="payload-pre-processing" containerID="cri-o://807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021" gracePeriod=30
Sep 10 13:32:24.633989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.633914    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:24.764846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.764822    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:32:24.767635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.767617    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:32:24.818064 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.818037    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e8eda21-6a5c-4543-ab8c-c67e87864524-plugins-config-volume\") pod \"9e8eda21-6a5c-4543-ab8c-c67e87864524\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") "
Sep 10 13:32:24.818181 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.818074    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vljql\" (UniqueName: \"kubernetes.io/projected/9e8eda21-6a5c-4543-ab8c-c67e87864524-kube-api-access-vljql\") pod \"9e8eda21-6a5c-4543-ab8c-c67e87864524\" (UID: \"9e8eda21-6a5c-4543-ab8c-c67e87864524\") "
Sep 10 13:32:24.818507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.818475    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8eda21-6a5c-4543-ab8c-c67e87864524-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e8eda21-6a5c-4543-ab8c-c67e87864524" (UID: "9e8eda21-6a5c-4543-ab8c-c67e87864524"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:32:24.820471 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.820442    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8eda21-6a5c-4543-ab8c-c67e87864524-kube-api-access-vljql" (OuterVolumeSpecName: "kube-api-access-vljql") pod "9e8eda21-6a5c-4543-ab8c-c67e87864524" (UID: "9e8eda21-6a5c-4543-ab8c-c67e87864524"). InnerVolumeSpecName "kube-api-access-vljql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:32:24.862649 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.862620    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:24.918774 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.918746    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-plugins-config-volume\") pod \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") "
Sep 10 13:32:24.918907 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.918802    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27ccr\" (UniqueName: \"kubernetes.io/projected/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-kube-api-access-27ccr\") pod \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\" (UID: \"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36\") "
Sep 10 13:32:24.919233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.919168    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e8eda21-6a5c-4543-ab8c-c67e87864524-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:32:24.919233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.919190    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vljql\" (UniqueName: \"kubernetes.io/projected/9e8eda21-6a5c-4543-ab8c-c67e87864524-kube-api-access-vljql\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:32:24.919418 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.919226    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" (UID: "8a184cf8-7ff2-4fa4-9cd8-5b80240fde36"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:32:24.920967 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.920946    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-kube-api-access-27ccr" (OuterVolumeSpecName: "kube-api-access-27ccr") pod "8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" (UID: "8a184cf8-7ff2-4fa4-9cd8-5b80240fde36"). InnerVolumeSpecName "kube-api-access-27ccr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:32:24.921278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.921260    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"]
Sep 10 13:32:24.934682 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:24.934632    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-d46083cd-2khn6"]
Sep 10 13:32:25.007446 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.007387    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:25.019595 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.019573    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:32:25.019595 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.019590    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27ccr\" (UniqueName: \"kubernetes.io/projected/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36-kube-api-access-27ccr\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:32:25.115065 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.115037    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:25.182549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.182524    2551 generic.go:356] "Generic (PLEG): container finished" podID="8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" containerID="807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021" exitCode=0
Sep 10 13:32:25.182660 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.182568    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"
Sep 10 13:32:25.182660 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.182624    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86" event={"ID":"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36","Type":"ContainerDied","Data":"807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021"}
Sep 10 13:32:25.182817 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.182699    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86" event={"ID":"8a184cf8-7ff2-4fa4-9cd8-5b80240fde36","Type":"ContainerDied","Data":"90e4c970530e77c12fbf7e49f1465c2e398341fab4649e436fa67ef45990b66a"}
Sep 10 13:32:25.182817 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.182732    2551 scope.go:117] "RemoveContainer" containerID="807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021"
Sep 10 13:32:25.184089 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.184065    2551 generic.go:356] "Generic (PLEG): container finished" podID="9e8eda21-6a5c-4543-ab8c-c67e87864524" containerID="df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858" exitCode=0
Sep 10 13:32:25.184196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.184100    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87" event={"ID":"9e8eda21-6a5c-4543-ab8c-c67e87864524","Type":"ContainerDied","Data":"df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858"}
Sep 10 13:32:25.184196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.184108    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"
Sep 10 13:32:25.184196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.184140    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87" event={"ID":"9e8eda21-6a5c-4543-ab8c-c67e87864524","Type":"ContainerDied","Data":"ec3a415483b62bb947593f0a48572070f89c16e9e27573700edc4dba1d2e7719"}
Sep 10 13:32:25.192277 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.192252    2551 scope.go:117] "RemoveContainer" containerID="807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021"
Sep 10 13:32:25.192539 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:25.192518    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021\": container with ID starting with 807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021 not found: ID does not exist" containerID="807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021"
Sep 10 13:32:25.192604 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.192546    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021"} err="failed to get container status \"807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021\": rpc error: code = NotFound desc = could not find container \"807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021\": container with ID starting with 807ebb05ead993b60f074c29828dc6ca660d7051ce1aa5aaf5de2e609f941021 not found: ID does not exist"
Sep 10 13:32:25.192604 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.192559    2551 scope.go:117] "RemoveContainer" containerID="df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858"
Sep 10 13:32:25.201996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.201977    2551 scope.go:117] "RemoveContainer" containerID="df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858"
Sep 10 13:32:25.202234 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:25.202217    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858\": container with ID starting with df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858 not found: ID does not exist" containerID="df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858"
Sep 10 13:32:25.202288 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.202240    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858"} err="failed to get container status \"df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858\": rpc error: code = NotFound desc = could not find container \"df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858\": container with ID starting with df7e03267366f931d6f1d1170de83a07d77935424004c60d96f30a654333b858 not found: ID does not exist"
Sep 10 13:32:25.214543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.214525    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"]
Sep 10 13:32:25.222303 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.222280    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-d46083cd-76fd8db9d5-bcn87"]
Sep 10 13:32:25.236638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.236621    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"]
Sep 10 13:32:25.238090 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.238072    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-d46083cd-77f55cf946-zdt86"]
Sep 10 13:32:25.247901 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.247883    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4561679e-0da6-4d92-8a23-6c90a5d21dcc" path="/var/lib/kubelet/pods/4561679e-0da6-4d92-8a23-6c90a5d21dcc/volumes"
Sep 10 13:32:25.248207 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.248196    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" path="/var/lib/kubelet/pods/8a184cf8-7ff2-4fa4-9cd8-5b80240fde36/volumes"
Sep 10 13:32:25.248491 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.248480    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8eda21-6a5c-4543-ab8c-c67e87864524" path="/var/lib/kubelet/pods/9e8eda21-6a5c-4543-ab8c-c67e87864524/volumes"
Sep 10 13:32:25.898405 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:25.898367    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:26.205823 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:26.205744    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:26.881547 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:26.881517    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:26.881724 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:26.881520    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:27.414678 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:27.414633    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:27.515046 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:27.515015    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:29.193215 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:29.193182    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:32:29.601379 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:29.601349    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:35.159589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:35.159557    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:35.328613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:35.328578    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:35.474866 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:35.474801    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:35.927617 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:35.927588    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:37.457149 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:37.457122    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:44.012961 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:44.012923    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:32:44.631195 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:44.631162    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:45.092559 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092525    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"]
Sep 10 13:32:45.092938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092878    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" containerName="payload-pre-processing"
Sep 10 13:32:45.092938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092890    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" containerName="payload-pre-processing"
Sep 10 13:32:45.092938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092901    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8eda21-6a5c-4543-ab8c-c67e87864524" containerName="payload-processing"
Sep 10 13:32:45.092938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092906    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8eda21-6a5c-4543-ab8c-c67e87864524" containerName="payload-processing"
Sep 10 13:32:45.092938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092915    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4561679e-0da6-4d92-8a23-6c90a5d21dcc" containerName="revoke-keys"
Sep 10 13:32:45.092938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092921    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4561679e-0da6-4d92-8a23-6c90a5d21dcc" containerName="revoke-keys"
Sep 10 13:32:45.093127 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092981    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8eda21-6a5c-4543-ab8c-c67e87864524" containerName="payload-processing"
Sep 10 13:32:45.093127 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092989    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4561679e-0da6-4d92-8a23-6c90a5d21dcc" containerName="revoke-keys"
Sep 10 13:32:45.093127 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.092997    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a184cf8-7ff2-4fa4-9cd8-5b80240fde36" containerName="payload-pre-processing"
Sep 10 13:32:45.095831 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.095814    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.100840 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.100819    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6beee4\"/\"default-dockercfg-jplgp\""
Sep 10 13:32:45.102286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.101969    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6beee4\"/\"kube-root-ca.crt\""
Sep 10 13:32:45.102286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.102054    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-6beee4\"/\"test-model-6beee4-kserve-self-signed-certs\""
Sep 10 13:32:45.102286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.102102    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-6beee4\"/\"openshift-service-ca.crt\""
Sep 10 13:32:45.120424 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.120403    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"]
Sep 10 13:32:45.155182 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.155152    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-model-cache\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.155302 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.155190    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tls-certs\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.155302 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.155231    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tmp-dir\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.155302 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.155259    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-home\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.155302 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.155278    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-dshm\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.155505 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.155302    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhzzv\" (UniqueName: \"kubernetes.io/projected/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-kube-api-access-jhzzv\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.190650 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.190625    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:45.255819 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.255797    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-model-cache\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.255919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.255834    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tls-certs\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.255919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.255864    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tmp-dir\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.255919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.255900    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-home\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.256103 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.255929    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-dshm\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.256103 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.255956    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhzzv\" (UniqueName: \"kubernetes.io/projected/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-kube-api-access-jhzzv\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.256234 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.256213    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-model-cache\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.256304 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.256248    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-home\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.256470 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.256450    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tmp-dir\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.258268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.258248    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-dshm\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.258429 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.258414    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tls-certs\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.266748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.266730    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhzzv\" (UniqueName: \"kubernetes.io/projected/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-kube-api-access-jhzzv\") pod \"test-model-6beee4-kserve-85f748cdd7-bh44w\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") " pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.404300 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.404216    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:45.686253 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:45.686228    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"]
Sep 10 13:32:45.688811 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:32:45.688777    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9925e78e_daf0_4675_bc84_b41cf1fb1dd9.slice/crio-951e056d67bdf7dc5f07387c56751e162c3fa012fe753d8d6d25025a9372b1e8 WatchSource:0}: Error finding container 951e056d67bdf7dc5f07387c56751e162c3fa012fe753d8d6d25025a9372b1e8: Status 404 returned error can't find the container with id 951e056d67bdf7dc5f07387c56751e162c3fa012fe753d8d6d25025a9372b1e8
Sep 10 13:32:46.282314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:46.282271    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w" event={"ID":"9925e78e-daf0-4675-bc84-b41cf1fb1dd9","Type":"ContainerStarted","Data":"e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178"}
Sep 10 13:32:46.282314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:46.282319    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w" event={"ID":"9925e78e-daf0-4675-bc84-b41cf1fb1dd9","Type":"ContainerStarted","Data":"951e056d67bdf7dc5f07387c56751e162c3fa012fe753d8d6d25025a9372b1e8"}
Sep 10 13:32:46.309651 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:46.309594    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w" podStartSLOduration=1.309579986 podStartE2EDuration="1.309579986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:32:46.307846586 +0000 UTC m=+931.519039843" watchObservedRunningTime="2026-09-10 13:32:46.309579986 +0000 UTC m=+931.520773243"
Sep 10 13:32:47.465220 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:47.465192    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:53.739269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.739230    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"]
Sep 10 13:32:53.742282 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.742261    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.747027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.747000    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-a26db0\"/\"explicit-model-a26db0-kserve-self-signed-certs\""
Sep 10 13:32:53.747160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.747037    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-a26db0\"/\"kube-root-ca.crt\""
Sep 10 13:32:53.747160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.747007    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-a26db0\"/\"default-dockercfg-k9hbq\""
Sep 10 13:32:53.747160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.747004    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-a26db0\"/\"openshift-service-ca.crt\""
Sep 10 13:32:53.761865 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.761841    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"]
Sep 10 13:32:53.780963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.780937    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:32:53.829783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.829760    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-dshm\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.829887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.829825    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20ebd605-709c-4436-8eac-a4d29b30f8ba-tls-certs\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.829933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.829903    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-tmp-dir\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.829969 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.829936    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-home\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.830007 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.829974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9czl\" (UniqueName: \"kubernetes.io/projected/20ebd605-709c-4436-8eac-a4d29b30f8ba-kube-api-access-x9czl\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.830007 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.830003    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-model-cache\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.930780 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.930753    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-home\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.930894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.930803    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9czl\" (UniqueName: \"kubernetes.io/projected/20ebd605-709c-4436-8eac-a4d29b30f8ba-kube-api-access-x9czl\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.930894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.930840    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-model-cache\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.930894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.930873    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-dshm\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.931063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.930928    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20ebd605-709c-4436-8eac-a4d29b30f8ba-tls-certs\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.931063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.930964    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-tmp-dir\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.931190 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.931172    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-home\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.931314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.931294    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-tmp-dir\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.931376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.931349    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-model-cache\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.932977 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.932957    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-dshm\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.933225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.933207    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20ebd605-709c-4436-8eac-a4d29b30f8ba-tls-certs\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:53.943066 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:53.943041    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9czl\" (UniqueName: \"kubernetes.io/projected/20ebd605-709c-4436-8eac-a4d29b30f8ba-kube-api-access-x9czl\") pod \"explicit-model-a26db0-kserve-69c585f579-qs5dn\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") " pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:54.052180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:54.052151    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:32:54.337097 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:54.337014    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"]
Sep 10 13:32:54.340724 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:32:54.340698    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20ebd605_709c_4436_8eac_a4d29b30f8ba.slice/crio-5d0b586bf2ce226cd9a6e89a47f7088b95acad89d36b07160b4f4d54a8c7780b WatchSource:0}: Error finding container 5d0b586bf2ce226cd9a6e89a47f7088b95acad89d36b07160b4f4d54a8c7780b: Status 404 returned error can't find the container with id 5d0b586bf2ce226cd9a6e89a47f7088b95acad89d36b07160b4f4d54a8c7780b
Sep 10 13:32:55.328452 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:55.328416    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn" event={"ID":"20ebd605-709c-4436-8eac-a4d29b30f8ba","Type":"ContainerStarted","Data":"ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399"}
Sep 10 13:32:55.328936 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:55.328467    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn" event={"ID":"20ebd605-709c-4436-8eac-a4d29b30f8ba","Type":"ContainerStarted","Data":"5d0b586bf2ce226cd9a6e89a47f7088b95acad89d36b07160b4f4d54a8c7780b"}
Sep 10 13:32:55.363227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:55.363181    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn" podStartSLOduration=2.363157604 podStartE2EDuration="2.363157604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:32:55.358469431 +0000 UTC m=+940.569662689" watchObservedRunningTime="2026-09-10 13:32:55.363157604 +0000 UTC m=+940.574350859"
Sep 10 13:32:55.404377 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:55.404351    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:55.404529 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:55.404395    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:55.418027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:55.418004    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:56.344072 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:56.344043    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:32:57.505931 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:57.505898    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:32:58.691995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:32:58.691960    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:32:59.605374 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:32:59.605332    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:33:01.912320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:01.912285    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:03.783565 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:03.783533    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"]
Sep 10 13:33:03.858802 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:03.858768    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:04.052659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:04.052574    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:33:04.219163 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:04.219124    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:07.547642 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:33:07.547607    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:33:10.367750 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:10.367715    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:10.497279 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:10.497241    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:10.674963 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:10.674865    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:10.894245 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:10.894210    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:33:14.637347 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:33:14.637310    2551 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4561679e_0da6_4d92_8a23_6c90a5d21dcc.slice\": RecentStats: unable to find data in memory cache]"
Sep 10 13:33:14.997574 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:14.997483    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:15.101536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.101502    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"]
Sep 10 13:33:15.101944 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.101881    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g" podUID="aaf0b995-d33a-469e-98c5-605baf08e80f" containerName="payload-processing" containerID="cri-o://a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17" gracePeriod=30
Sep 10 13:33:15.135759 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.135728    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"]
Sep 10 13:33:15.135944 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.135923    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c" podUID="0b62ede4-a34f-43a9-b11a-883efa72889f" containerName="payload-pre-processing" containerID="cri-o://44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce" gracePeriod=30
Sep 10 13:33:15.211063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.211034    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:15.371174 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.371139    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:15.427719 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.427694    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:33:15.435443 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.435193    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:33:15.438538 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.438508    2551 generic.go:356] "Generic (PLEG): container finished" podID="aaf0b995-d33a-469e-98c5-605baf08e80f" containerID="a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17" exitCode=0
Sep 10 13:33:15.438657 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.438561    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"
Sep 10 13:33:15.438657 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.438595    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g" event={"ID":"aaf0b995-d33a-469e-98c5-605baf08e80f","Type":"ContainerDied","Data":"a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17"}
Sep 10 13:33:15.438792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.438691    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g" event={"ID":"aaf0b995-d33a-469e-98c5-605baf08e80f","Type":"ContainerDied","Data":"eaf59ed5e0f1957c530d55e3ea06ba61c689d5a5c848153735ac5b258bab21c1"}
Sep 10 13:33:15.438792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.438724    2551 scope.go:117] "RemoveContainer" containerID="a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17"
Sep 10 13:33:15.439935 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.439918    2551 generic.go:356] "Generic (PLEG): container finished" podID="0b62ede4-a34f-43a9-b11a-883efa72889f" containerID="44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce" exitCode=0
Sep 10 13:33:15.440008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.439954    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c" event={"ID":"0b62ede4-a34f-43a9-b11a-883efa72889f","Type":"ContainerDied","Data":"44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce"}
Sep 10 13:33:15.440008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.439968    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"
Sep 10 13:33:15.440008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.439983    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c" event={"ID":"0b62ede4-a34f-43a9-b11a-883efa72889f","Type":"ContainerDied","Data":"1d9063324d87982f77871107af0d49bdaad5dc79405e62c2814a0c1542d893f1"}
Sep 10 13:33:15.447237 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.447218    2551 scope.go:117] "RemoveContainer" containerID="a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17"
Sep 10 13:33:15.447527 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:33:15.447508    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17\": container with ID starting with a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17 not found: ID does not exist" containerID="a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17"
Sep 10 13:33:15.447570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.447536    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17"} err="failed to get container status \"a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17\": rpc error: code = NotFound desc = could not find container \"a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17\": container with ID starting with a980f93aada604776223e72b44bbb4b1378513770c1cd5bca209409ed62a9d17 not found: ID does not exist"
Sep 10 13:33:15.447570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.447553    2551 scope.go:117] "RemoveContainer" containerID="44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce"
Sep 10 13:33:15.456102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.456081    2551 scope.go:117] "RemoveContainer" containerID="44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce"
Sep 10 13:33:15.456440 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:33:15.456415    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce\": container with ID starting with 44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce not found: ID does not exist" containerID="44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce"
Sep 10 13:33:15.456493 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.456453    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce"} err="failed to get container status \"44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce\": rpc error: code = NotFound desc = could not find container \"44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce\": container with ID starting with 44764585395c765f0285411116d36895a695e436d3d426017d9a6f9c87e098ce not found: ID does not exist"
Sep 10 13:33:15.521607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.521575    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74z77\" (UniqueName: \"kubernetes.io/projected/aaf0b995-d33a-469e-98c5-605baf08e80f-kube-api-access-74z77\") pod \"aaf0b995-d33a-469e-98c5-605baf08e80f\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") "
Sep 10 13:33:15.521607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.521611    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64p9m\" (UniqueName: \"kubernetes.io/projected/0b62ede4-a34f-43a9-b11a-883efa72889f-kube-api-access-64p9m\") pod \"0b62ede4-a34f-43a9-b11a-883efa72889f\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") "
Sep 10 13:33:15.521855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.521657    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaf0b995-d33a-469e-98c5-605baf08e80f-plugins-config-volume\") pod \"aaf0b995-d33a-469e-98c5-605baf08e80f\" (UID: \"aaf0b995-d33a-469e-98c5-605baf08e80f\") "
Sep 10 13:33:15.521855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.521701    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b62ede4-a34f-43a9-b11a-883efa72889f-plugins-config-volume\") pod \"0b62ede4-a34f-43a9-b11a-883efa72889f\" (UID: \"0b62ede4-a34f-43a9-b11a-883efa72889f\") "
Sep 10 13:33:15.522121 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.522096    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaf0b995-d33a-469e-98c5-605baf08e80f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aaf0b995-d33a-469e-98c5-605baf08e80f" (UID: "aaf0b995-d33a-469e-98c5-605baf08e80f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:33:15.522188 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.522118    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b62ede4-a34f-43a9-b11a-883efa72889f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0b62ede4-a34f-43a9-b11a-883efa72889f" (UID: "0b62ede4-a34f-43a9-b11a-883efa72889f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:33:15.524166 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.524135    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b62ede4-a34f-43a9-b11a-883efa72889f-kube-api-access-64p9m" (OuterVolumeSpecName: "kube-api-access-64p9m") pod "0b62ede4-a34f-43a9-b11a-883efa72889f" (UID: "0b62ede4-a34f-43a9-b11a-883efa72889f"). InnerVolumeSpecName "kube-api-access-64p9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:33:15.524294 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.524204    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf0b995-d33a-469e-98c5-605baf08e80f-kube-api-access-74z77" (OuterVolumeSpecName: "kube-api-access-74z77") pod "aaf0b995-d33a-469e-98c5-605baf08e80f" (UID: "aaf0b995-d33a-469e-98c5-605baf08e80f"). InnerVolumeSpecName "kube-api-access-74z77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:33:15.534033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.534002    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:15.622620 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.622578    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74z77\" (UniqueName: \"kubernetes.io/projected/aaf0b995-d33a-469e-98c5-605baf08e80f-kube-api-access-74z77\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:15.622620 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.622612    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64p9m\" (UniqueName: \"kubernetes.io/projected/0b62ede4-a34f-43a9-b11a-883efa72889f-kube-api-access-64p9m\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:15.622620 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.622627    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aaf0b995-d33a-469e-98c5-605baf08e80f-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:15.622907 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.622640    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b62ede4-a34f-43a9-b11a-883efa72889f-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:15.782196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.782107    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"]
Sep 10 13:33:15.791127 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.791096    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-a26db0-65b9875df4-psn7c"]
Sep 10 13:33:15.814010 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.813970    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"]
Sep 10 13:33:15.824536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.824503    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-a26db0-5845956bb5-2rj7g"]
Sep 10 13:33:15.842099 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:15.842071    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:16.028198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:16.028166    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:17.248685 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:17.248642    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b62ede4-a34f-43a9-b11a-883efa72889f" path="/var/lib/kubelet/pods/0b62ede4-a34f-43a9-b11a-883efa72889f/volumes"
Sep 10 13:33:17.249061 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:17.249012    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaf0b995-d33a-469e-98c5-605baf08e80f" path="/var/lib/kubelet/pods/aaf0b995-d33a-469e-98c5-605baf08e80f/volumes"
Sep 10 13:33:17.576237 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:17.576205    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:18.802794 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:18.802749    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn" podUID="20ebd605-709c-4436-8eac-a4d29b30f8ba" containerName="main" containerID="cri-o://ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399" gracePeriod=45
Sep 10 13:33:19.052990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.052925    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:33:19.154564 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154525    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-model-cache\") pod \"20ebd605-709c-4436-8eac-a4d29b30f8ba\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") "
Sep 10 13:33:19.154564 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154567    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-tmp-dir\") pod \"20ebd605-709c-4436-8eac-a4d29b30f8ba\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") "
Sep 10 13:33:19.154800 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154647    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9czl\" (UniqueName: \"kubernetes.io/projected/20ebd605-709c-4436-8eac-a4d29b30f8ba-kube-api-access-x9czl\") pod \"20ebd605-709c-4436-8eac-a4d29b30f8ba\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") "
Sep 10 13:33:19.154800 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154721    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20ebd605-709c-4436-8eac-a4d29b30f8ba-tls-certs\") pod \"20ebd605-709c-4436-8eac-a4d29b30f8ba\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") "
Sep 10 13:33:19.154891 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154865    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-home\") pod \"20ebd605-709c-4436-8eac-a4d29b30f8ba\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") "
Sep 10 13:33:19.154934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154914    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-dshm\") pod \"20ebd605-709c-4436-8eac-a4d29b30f8ba\" (UID: \"20ebd605-709c-4436-8eac-a4d29b30f8ba\") "
Sep 10 13:33:19.154934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154875    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "20ebd605-709c-4436-8eac-a4d29b30f8ba" (UID: "20ebd605-709c-4436-8eac-a4d29b30f8ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:19.155010 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.154917    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20ebd605-709c-4436-8eac-a4d29b30f8ba" (UID: "20ebd605-709c-4436-8eac-a4d29b30f8ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:19.155089 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.155069    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-home" (OuterVolumeSpecName: "home") pod "20ebd605-709c-4436-8eac-a4d29b30f8ba" (UID: "20ebd605-709c-4436-8eac-a4d29b30f8ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:19.155190 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.155177    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:19.155256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.155192    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:19.155256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.155202    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:19.157040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.157004    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20ebd605-709c-4436-8eac-a4d29b30f8ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20ebd605-709c-4436-8eac-a4d29b30f8ba" (UID: "20ebd605-709c-4436-8eac-a4d29b30f8ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:33:19.157160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.157057    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-dshm" (OuterVolumeSpecName: "dshm") pod "20ebd605-709c-4436-8eac-a4d29b30f8ba" (UID: "20ebd605-709c-4436-8eac-a4d29b30f8ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:19.157160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.157061    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20ebd605-709c-4436-8eac-a4d29b30f8ba-kube-api-access-x9czl" (OuterVolumeSpecName: "kube-api-access-x9czl") pod "20ebd605-709c-4436-8eac-a4d29b30f8ba" (UID: "20ebd605-709c-4436-8eac-a4d29b30f8ba"). InnerVolumeSpecName "kube-api-access-x9czl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:33:19.255640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.255613    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9czl\" (UniqueName: \"kubernetes.io/projected/20ebd605-709c-4436-8eac-a4d29b30f8ba-kube-api-access-x9czl\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:19.255640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.255639    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20ebd605-709c-4436-8eac-a4d29b30f8ba-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:19.255805 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.255649    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20ebd605-709c-4436-8eac-a4d29b30f8ba-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:19.462280 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.462192    2551 generic.go:356] "Generic (PLEG): container finished" podID="20ebd605-709c-4436-8eac-a4d29b30f8ba" containerID="ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399" exitCode=0
Sep 10 13:33:19.462280 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.462240    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"
Sep 10 13:33:19.462475 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.462272    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn" event={"ID":"20ebd605-709c-4436-8eac-a4d29b30f8ba","Type":"ContainerDied","Data":"ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399"}
Sep 10 13:33:19.462475 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.462328    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn" event={"ID":"20ebd605-709c-4436-8eac-a4d29b30f8ba","Type":"ContainerDied","Data":"5d0b586bf2ce226cd9a6e89a47f7088b95acad89d36b07160b4f4d54a8c7780b"}
Sep 10 13:33:19.462475 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.462347    2551 scope.go:117] "RemoveContainer" containerID="ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399"
Sep 10 13:33:19.470768 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.470751    2551 scope.go:117] "RemoveContainer" containerID="ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399"
Sep 10 13:33:19.471046 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:33:19.471028    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399\": container with ID starting with ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399 not found: ID does not exist" containerID="ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399"
Sep 10 13:33:19.471100 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.471056    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399"} err="failed to get container status \"ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399\": rpc error: code = NotFound desc = could not find container \"ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399\": container with ID starting with ec89095e5ab4188b5e56d63fcdd26e9a1ce2c26f0c1fa981e48e571ae19ff399 not found: ID does not exist"
Sep 10 13:33:19.491413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.491384    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"]
Sep 10 13:33:19.497648 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:19.497622    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-a26db0/explicit-model-a26db0-kserve-69c585f579-qs5dn"]
Sep 10 13:33:21.248145 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:21.248108    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20ebd605-709c-4436-8eac-a4d29b30f8ba" path="/var/lib/kubelet/pods/20ebd605-709c-4436-8eac-a4d29b30f8ba/volumes"
Sep 10 13:33:22.317421 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:22.317390    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:33:23.759540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:23.759502    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:23.864878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:23.864845    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:24.142826 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:24.142786    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:24.615309 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:24.615274    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:25.029920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:25.029892    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:25.278930 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:25.278897    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:26.462379 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:26.462346    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:26.633384 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:26.633348    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:26.793679 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:26.793637    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:33.248757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.248718    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"]
Sep 10 13:33:33.249226 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249207    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20ebd605-709c-4436-8eac-a4d29b30f8ba" containerName="main"
Sep 10 13:33:33.249308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249229    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="20ebd605-709c-4436-8eac-a4d29b30f8ba" containerName="main"
Sep 10 13:33:33.249308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249245    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf0b995-d33a-469e-98c5-605baf08e80f" containerName="payload-processing"
Sep 10 13:33:33.249308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249253    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf0b995-d33a-469e-98c5-605baf08e80f" containerName="payload-processing"
Sep 10 13:33:33.249466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249307    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b62ede4-a34f-43a9-b11a-883efa72889f" containerName="payload-pre-processing"
Sep 10 13:33:33.249466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249318    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b62ede4-a34f-43a9-b11a-883efa72889f" containerName="payload-pre-processing"
Sep 10 13:33:33.249466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249394    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="20ebd605-709c-4436-8eac-a4d29b30f8ba" containerName="main"
Sep 10 13:33:33.249466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249406    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaf0b995-d33a-469e-98c5-605baf08e80f" containerName="payload-processing"
Sep 10 13:33:33.249466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.249420    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b62ede4-a34f-43a9-b11a-883efa72889f" containerName="payload-pre-processing"
Sep 10 13:33:33.253770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.253750    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.256845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.256824    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-669b24\"/\"kube-root-ca.crt\""
Sep 10 13:33:33.258084 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.258066    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-669b24\"/\"openshift-service-ca.crt\""
Sep 10 13:33:33.258180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.258106    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-669b24\"/\"default-dockercfg-vh4v8\""
Sep 10 13:33:33.258180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.258174    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-669b24\"/\"noten-model-669b24-kserve-self-signed-certs\""
Sep 10 13:33:33.261678 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.261646    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:33.266334 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.266310    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"]
Sep 10 13:33:33.273022 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.272996    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-model-cache\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.273137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.273035    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-tmp-dir\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.273137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.273096    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-home\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.273137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.273125    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2f7bf16-dce0-4518-a84e-2265c4943c33-tls-certs\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.273291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.273166    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-dshm\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.273291 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.273188    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpzhg\" (UniqueName: \"kubernetes.io/projected/c2f7bf16-dce0-4518-a84e-2265c4943c33-kube-api-access-lpzhg\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.373559 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.373526    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-home\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.373702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.373572    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2f7bf16-dce0-4518-a84e-2265c4943c33-tls-certs\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.373702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.373589    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-dshm\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.373702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.373610    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpzhg\" (UniqueName: \"kubernetes.io/projected/c2f7bf16-dce0-4518-a84e-2265c4943c33-kube-api-access-lpzhg\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.373702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.373647    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-model-cache\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.373936 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.373836    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-tmp-dir\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.374035 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.374015    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-model-cache\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.374101 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.374084    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-home\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.374186 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.374172    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-tmp-dir\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.375928 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.375902    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-dshm\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.376126 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.376111    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2f7bf16-dce0-4518-a84e-2265c4943c33-tls-certs\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.382816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.382795    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpzhg\" (UniqueName: \"kubernetes.io/projected/c2f7bf16-dce0-4518-a84e-2265c4943c33-kube-api-access-lpzhg\") pod \"noten-model-669b24-kserve-77564d5467-gv88f\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") " pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.563296 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.563267    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:33.835704 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:33.835682    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"]
Sep 10 13:33:33.837449 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:33:33.837417    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2f7bf16_dce0_4518_a84e_2265c4943c33.slice/crio-e5bb1b439f98b4816c231cd8aff1f85bf57d3a72cd50c741edfc3dee09b5af5a WatchSource:0}: Error finding container e5bb1b439f98b4816c231cd8aff1f85bf57d3a72cd50c741edfc3dee09b5af5a: Status 404 returned error can't find the container with id e5bb1b439f98b4816c231cd8aff1f85bf57d3a72cd50c741edfc3dee09b5af5a
Sep 10 13:33:34.540593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:34.540555    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f" event={"ID":"c2f7bf16-dce0-4518-a84e-2265c4943c33","Type":"ContainerStarted","Data":"40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0"}
Sep 10 13:33:34.540593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:34.540599    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f" event={"ID":"c2f7bf16-dce0-4518-a84e-2265c4943c33","Type":"ContainerStarted","Data":"e5bb1b439f98b4816c231cd8aff1f85bf57d3a72cd50c741edfc3dee09b5af5a"}
Sep 10 13:33:34.605892 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:34.605847    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f" podStartSLOduration=1.605831973 podStartE2EDuration="1.605831973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:33:34.603496369 +0000 UTC m=+979.814689627" watchObservedRunningTime="2026-09-10 13:33:34.605831973 +0000 UTC m=+979.817025229"
Sep 10 13:33:34.872182 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:34.872105    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:34.966271 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:34.966234    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:39.099575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:39.099495    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"]
Sep 10 13:33:39.299816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:39.299786    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:39.400869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:39.400795    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:39.501014 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:39.500992    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:39.603751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:39.603717    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:43.563973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:43.563930    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:44.343040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:44.343003    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:44.683716 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:44.683614    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:45.202287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:45.202250    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:52.017382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:52.017339    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:52.085725 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:52.085683    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:52.189203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:52.189169    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:52.291934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:52.291904    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:53.082237 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:53.082202    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:53.223960 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:53.223928    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:53.323451 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:53.323394    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:54.118841 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.118794    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f" podUID="c2f7bf16-dce0-4518-a84e-2265c4943c33" containerName="main" containerID="cri-o://40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0" gracePeriod=45
Sep 10 13:33:54.362218 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.362192    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:54.443798 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.443714    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-home\") pod \"c2f7bf16-dce0-4518-a84e-2265c4943c33\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") "
Sep 10 13:33:54.443958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.443811    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-tmp-dir\") pod \"c2f7bf16-dce0-4518-a84e-2265c4943c33\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") "
Sep 10 13:33:54.443958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.443829    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-model-cache\") pod \"c2f7bf16-dce0-4518-a84e-2265c4943c33\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") "
Sep 10 13:33:54.443958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.443857    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-dshm\") pod \"c2f7bf16-dce0-4518-a84e-2265c4943c33\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") "
Sep 10 13:33:54.443958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.443893    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpzhg\" (UniqueName: \"kubernetes.io/projected/c2f7bf16-dce0-4518-a84e-2265c4943c33-kube-api-access-lpzhg\") pod \"c2f7bf16-dce0-4518-a84e-2265c4943c33\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") "
Sep 10 13:33:54.443958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.443916    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2f7bf16-dce0-4518-a84e-2265c4943c33-tls-certs\") pod \"c2f7bf16-dce0-4518-a84e-2265c4943c33\" (UID: \"c2f7bf16-dce0-4518-a84e-2265c4943c33\") "
Sep 10 13:33:54.444228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.443976    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-home" (OuterVolumeSpecName: "home") pod "c2f7bf16-dce0-4518-a84e-2265c4943c33" (UID: "c2f7bf16-dce0-4518-a84e-2265c4943c33"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:54.444228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.444122    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2f7bf16-dce0-4518-a84e-2265c4943c33" (UID: "c2f7bf16-dce0-4518-a84e-2265c4943c33"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:54.444228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.444128    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2f7bf16-dce0-4518-a84e-2265c4943c33" (UID: "c2f7bf16-dce0-4518-a84e-2265c4943c33"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:54.444228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.444204    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:54.444228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.444220    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:54.446090 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.446066    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-dshm" (OuterVolumeSpecName: "dshm") pod "c2f7bf16-dce0-4518-a84e-2265c4943c33" (UID: "c2f7bf16-dce0-4518-a84e-2265c4943c33"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:33:54.446177 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.446069    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2f7bf16-dce0-4518-a84e-2265c4943c33-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2f7bf16-dce0-4518-a84e-2265c4943c33" (UID: "c2f7bf16-dce0-4518-a84e-2265c4943c33"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:33:54.446177 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.446121    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2f7bf16-dce0-4518-a84e-2265c4943c33-kube-api-access-lpzhg" (OuterVolumeSpecName: "kube-api-access-lpzhg") pod "c2f7bf16-dce0-4518-a84e-2265c4943c33" (UID: "c2f7bf16-dce0-4518-a84e-2265c4943c33"). InnerVolumeSpecName "kube-api-access-lpzhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:33:54.544912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.544868    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:54.544912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.544901    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2f7bf16-dce0-4518-a84e-2265c4943c33-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:54.544912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.544911    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpzhg\" (UniqueName: \"kubernetes.io/projected/c2f7bf16-dce0-4518-a84e-2265c4943c33-kube-api-access-lpzhg\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:54.544912 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.544924    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2f7bf16-dce0-4518-a84e-2265c4943c33-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:33:54.647226 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.647189    2551 generic.go:356] "Generic (PLEG): container finished" podID="c2f7bf16-dce0-4518-a84e-2265c4943c33" containerID="40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0" exitCode=0
Sep 10 13:33:54.647417 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.647235    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"
Sep 10 13:33:54.647417 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.647275    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f" event={"ID":"c2f7bf16-dce0-4518-a84e-2265c4943c33","Type":"ContainerDied","Data":"40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0"}
Sep 10 13:33:54.647417 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.647316    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f" event={"ID":"c2f7bf16-dce0-4518-a84e-2265c4943c33","Type":"ContainerDied","Data":"e5bb1b439f98b4816c231cd8aff1f85bf57d3a72cd50c741edfc3dee09b5af5a"}
Sep 10 13:33:54.647417 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.647335    2551 scope.go:117] "RemoveContainer" containerID="40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0"
Sep 10 13:33:54.656950 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.656933    2551 scope.go:117] "RemoveContainer" containerID="40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0"
Sep 10 13:33:54.657210 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:33:54.657191    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0\": container with ID starting with 40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0 not found: ID does not exist" containerID="40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0"
Sep 10 13:33:54.657275 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.657222    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0"} err="failed to get container status \"40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0\": rpc error: code = NotFound desc = could not find container \"40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0\": container with ID starting with 40907537ecf796c7f82d8b01941d6f095f2c6978651637f80da21a2b5bb752a0 not found: ID does not exist"
Sep 10 13:33:54.672188 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.672160    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"]
Sep 10 13:33:54.675363 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.675342    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-669b24/noten-model-669b24-kserve-77564d5467-gv88f"]
Sep 10 13:33:54.891573 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:54.891539    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:55.249279 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:55.249182    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2f7bf16-dce0-4518-a84e-2265c4943c33" path="/var/lib/kubelet/pods/c2f7bf16-dce0-4518-a84e-2265c4943c33/volumes"
Sep 10 13:33:57.697117 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:57.697083    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:33:58.126231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:58.126192    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:59.114197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.114167    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"]
Sep 10 13:33:59.140199 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.140132    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:59.238747 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.238708    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"]
Sep 10 13:33:59.239156 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.239138    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2f7bf16-dce0-4518-a84e-2265c4943c33" containerName="main"
Sep 10 13:33:59.239252 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.239159    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2f7bf16-dce0-4518-a84e-2265c4943c33" containerName="main"
Sep 10 13:33:59.239252 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.239237    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2f7bf16-dce0-4518-a84e-2265c4943c33" containerName="main"
Sep 10 13:33:59.243736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.243707    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"
Sep 10 13:33:59.246873 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.246850    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:33:59.246986 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.246859    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:33:59.250845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.250814    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"]
Sep 10 13:33:59.385280 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.385193    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d88650-1263-461c-8971-82828101fccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f\" (UID: \"f0d88650-1263-461c-8971-82828101fccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"
Sep 10 13:33:59.385280 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.385266    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:33:59.486255 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.486217    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d88650-1263-461c-8971-82828101fccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f\" (UID: \"f0d88650-1263-461c-8971-82828101fccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"
Sep 10 13:33:59.486930 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.486910    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d88650-1263-461c-8971-82828101fccc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f\" (UID: \"f0d88650-1263-461c-8971-82828101fccc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"
Sep 10 13:33:59.556380 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.556347    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"
Sep 10 13:33:59.828757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:33:59.828731    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"]
Sep 10 13:33:59.831105 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:33:59.831076    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d88650_1263_461c_8971_82828101fccc.slice/crio-1d1d7ae8485200b36ffee10003a325524d2f021fc7d1690aeb4e7aa13b8f7132 WatchSource:0}: Error finding container 1d1d7ae8485200b36ffee10003a325524d2f021fc7d1690aeb4e7aa13b8f7132: Status 404 returned error can't find the container with id 1d1d7ae8485200b36ffee10003a325524d2f021fc7d1690aeb4e7aa13b8f7132
Sep 10 13:34:00.678464 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:00.678432    2551 generic.go:356] "Generic (PLEG): container finished" podID="f0d88650-1263-461c-8971-82828101fccc" containerID="87eb749be0ad681313fb0f7b87c7985a8dc1d35db7e068a448bee30528eabe1a" exitCode=0
Sep 10 13:34:00.678814 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:00.678520    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f" event={"ID":"f0d88650-1263-461c-8971-82828101fccc","Type":"ContainerDied","Data":"87eb749be0ad681313fb0f7b87c7985a8dc1d35db7e068a448bee30528eabe1a"}
Sep 10 13:34:00.678814 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:00.678579    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f" event={"ID":"f0d88650-1263-461c-8971-82828101fccc","Type":"ContainerStarted","Data":"1d1d7ae8485200b36ffee10003a325524d2f021fc7d1690aeb4e7aa13b8f7132"}
Sep 10 13:34:01.804120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:01.804097    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"
Sep 10 13:34:01.905233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:01.905209    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d88650-1263-461c-8971-82828101fccc-maas-api-service-ca\") pod \"f0d88650-1263-461c-8971-82828101fccc\" (UID: \"f0d88650-1263-461c-8971-82828101fccc\") "
Sep 10 13:34:01.905552 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:01.905528    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0d88650-1263-461c-8971-82828101fccc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f0d88650-1263-461c-8971-82828101fccc" (UID: "f0d88650-1263-461c-8971-82828101fccc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:34:01.963056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:01.962992    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:02.006248 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:02.006226    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d88650-1263-461c-8971-82828101fccc-maas-api-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:02.029851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:02.029824    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:02.675320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:02.675284    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:02.688860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:02.688840    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"
Sep 10 13:34:02.688970 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:02.688838    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f" event={"ID":"f0d88650-1263-461c-8971-82828101fccc","Type":"ContainerDied","Data":"1d1d7ae8485200b36ffee10003a325524d2f021fc7d1690aeb4e7aa13b8f7132"}
Sep 10 13:34:02.688970 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:02.688964    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d1d7ae8485200b36ffee10003a325524d2f021fc7d1690aeb4e7aa13b8f7132"
Sep 10 13:34:09.431309 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:09.431270    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:09.532502 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:09.532439    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:09.710928 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:09.710845    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:09.994919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:09.994835    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:10.121470 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:10.121421    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:10.241320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:10.241273    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:10.487741 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:10.487710    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:10.541818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:10.541787    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"]
Sep 10 13:34:10.545094 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:10.545070    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-39df6e-nh84f"]
Sep 10 13:34:10.560176 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:10.560144    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:10.927776 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:10.927745    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:11.248136 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:11.248038    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0d88650-1263-461c-8971-82828101fccc" path="/var/lib/kubelet/pods/f0d88650-1263-461c-8971-82828101fccc/volumes"
Sep 10 13:34:12.306889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:12.306858    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:34:13.089878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:13.089843    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:13.169299 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:13.169266    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:14.133411 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.133367    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w" podUID="9925e78e-daf0-4675-bc84-b41cf1fb1dd9" containerName="main" containerID="cri-o://e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178" gracePeriod=45
Sep 10 13:34:14.280557 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.280504    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:14.371285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.371246    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:14.385460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.385399    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:34:14.460684 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.460645    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:14.507764 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.507734    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-home\") pod \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") "
Sep 10 13:34:14.507956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.507825    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tls-certs\") pod \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") "
Sep 10 13:34:14.507956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.507863    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-dshm\") pod \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") "
Sep 10 13:34:14.507956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.507889    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhzzv\" (UniqueName: \"kubernetes.io/projected/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-kube-api-access-jhzzv\") pod \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") "
Sep 10 13:34:14.508135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508027    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-home" (OuterVolumeSpecName: "home") pod "9925e78e-daf0-4675-bc84-b41cf1fb1dd9" (UID: "9925e78e-daf0-4675-bc84-b41cf1fb1dd9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:34:14.508135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508040    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-model-cache\") pod \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") "
Sep 10 13:34:14.508135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508120    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tmp-dir\") pod \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\" (UID: \"9925e78e-daf0-4675-bc84-b41cf1fb1dd9\") "
Sep 10 13:34:14.508289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508225    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-model-cache" (OuterVolumeSpecName: "model-cache") pod "9925e78e-daf0-4675-bc84-b41cf1fb1dd9" (UID: "9925e78e-daf0-4675-bc84-b41cf1fb1dd9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:34:14.508373 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508349    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9925e78e-daf0-4675-bc84-b41cf1fb1dd9" (UID: "9925e78e-daf0-4675-bc84-b41cf1fb1dd9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:34:14.508463 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508449    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:14.508506 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508472    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:14.508506 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.508488    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:14.510178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.510154    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9925e78e-daf0-4675-bc84-b41cf1fb1dd9" (UID: "9925e78e-daf0-4675-bc84-b41cf1fb1dd9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:34:14.510289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.510177    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-dshm" (OuterVolumeSpecName: "dshm") pod "9925e78e-daf0-4675-bc84-b41cf1fb1dd9" (UID: "9925e78e-daf0-4675-bc84-b41cf1fb1dd9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:34:14.510289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.510225    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-kube-api-access-jhzzv" (OuterVolumeSpecName: "kube-api-access-jhzzv") pod "9925e78e-daf0-4675-bc84-b41cf1fb1dd9" (UID: "9925e78e-daf0-4675-bc84-b41cf1fb1dd9"). InnerVolumeSpecName "kube-api-access-jhzzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:34:14.608894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.608850    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:14.608894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.608877    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:14.608894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.608885    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhzzv\" (UniqueName: \"kubernetes.io/projected/9925e78e-daf0-4675-bc84-b41cf1fb1dd9-kube-api-access-jhzzv\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:14.746032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.745947    2551 generic.go:356] "Generic (PLEG): container finished" podID="9925e78e-daf0-4675-bc84-b41cf1fb1dd9" containerID="e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178" exitCode=0
Sep 10 13:34:14.746032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.745996    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"
Sep 10 13:34:14.746218 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.745993    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w" event={"ID":"9925e78e-daf0-4675-bc84-b41cf1fb1dd9","Type":"ContainerDied","Data":"e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178"}
Sep 10 13:34:14.746218 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.746107    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w" event={"ID":"9925e78e-daf0-4675-bc84-b41cf1fb1dd9","Type":"ContainerDied","Data":"951e056d67bdf7dc5f07387c56751e162c3fa012fe753d8d6d25025a9372b1e8"}
Sep 10 13:34:14.746218 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.746133    2551 scope.go:117] "RemoveContainer" containerID="e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178"
Sep 10 13:34:14.755773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.755752    2551 scope.go:117] "RemoveContainer" containerID="e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178"
Sep 10 13:34:14.756084 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:34:14.756062    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178\": container with ID starting with e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178 not found: ID does not exist" containerID="e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178"
Sep 10 13:34:14.756137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.756098    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178"} err="failed to get container status \"e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178\": rpc error: code = NotFound desc = could not find container \"e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178\": container with ID starting with e17491861041e3fc448b5807bdbbeecf5f39d24bce636e82f0618ac6799d3178 not found: ID does not exist"
Sep 10 13:34:14.771993 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.771968    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"]
Sep 10 13:34:14.777882 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.777856    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-6beee4/test-model-6beee4-kserve-85f748cdd7-bh44w"]
Sep 10 13:34:14.849211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.849147    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:14.951630 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.951589    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:14.986943 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.986907    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"]
Sep 10 13:34:14.987270 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.987258    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d88650-1263-461c-8971-82828101fccc" containerName="revoke-keys"
Sep 10 13:34:14.987315 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.987273    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d88650-1263-461c-8971-82828101fccc" containerName="revoke-keys"
Sep 10 13:34:14.987315 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.987295    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9925e78e-daf0-4675-bc84-b41cf1fb1dd9" containerName="main"
Sep 10 13:34:14.987315 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.987300    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9925e78e-daf0-4675-bc84-b41cf1fb1dd9" containerName="main"
Sep 10 13:34:14.987408 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.987354    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0d88650-1263-461c-8971-82828101fccc" containerName="revoke-keys"
Sep 10 13:34:14.987408 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.987363    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9925e78e-daf0-4675-bc84-b41cf1fb1dd9" containerName="main"
Sep 10 13:34:14.991678 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.991649    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:14.994752 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:14.994730    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7f8babf4\""
Sep 10 13:34:15.017796 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.017769    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"]
Sep 10 13:34:15.113293 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.113261    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beeb1520-a721-44c7-802d-aa2d7a00d074-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.113477 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.113318    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbbz8\" (UniqueName: \"kubernetes.io/projected/beeb1520-a721-44c7-802d-aa2d7a00d074-kube-api-access-rbbz8\") pod \"payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.214840 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.214798    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbbz8\" (UniqueName: \"kubernetes.io/projected/beeb1520-a721-44c7-802d-aa2d7a00d074-kube-api-access-rbbz8\") pod \"payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.215287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.214903    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beeb1520-a721-44c7-802d-aa2d7a00d074-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.218414 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.218394    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7f8babf4\""
Sep 10 13:34:15.226582 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.226555    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beeb1520-a721-44c7-802d-aa2d7a00d074-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.231822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.231799    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbbz8\" (UniqueName: \"kubernetes.io/projected/beeb1520-a721-44c7-802d-aa2d7a00d074-kube-api-access-rbbz8\") pod \"payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.257436 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.257353    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9925e78e-daf0-4675-bc84-b41cf1fb1dd9" path="/var/lib/kubelet/pods/9925e78e-daf0-4675-bc84-b41cf1fb1dd9/volumes"
Sep 10 13:34:15.301286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.301263    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.590298 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.590260    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"]
Sep 10 13:34:15.593508 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:34:15.593472    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeeb1520_a721_44c7_802d_aa2d7a00d074.slice/crio-49e0a101aeec0a5b0bb6a31f7bb7bedf001fb66beba9bf43e64c94b7675e77a4 WatchSource:0}: Error finding container 49e0a101aeec0a5b0bb6a31f7bb7bedf001fb66beba9bf43e64c94b7675e77a4: Status 404 returned error can't find the container with id 49e0a101aeec0a5b0bb6a31f7bb7bedf001fb66beba9bf43e64c94b7675e77a4
Sep 10 13:34:15.752603 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.752569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v" event={"ID":"beeb1520-a721-44c7-802d-aa2d7a00d074","Type":"ContainerStarted","Data":"0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2"}
Sep 10 13:34:15.752818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.752628    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v" event={"ID":"beeb1520-a721-44c7-802d-aa2d7a00d074","Type":"ContainerStarted","Data":"49e0a101aeec0a5b0bb6a31f7bb7bedf001fb66beba9bf43e64c94b7675e77a4"}
Sep 10 13:34:15.752818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.752658    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:15.798014 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:15.797956    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v" podStartSLOduration=1.797939134 podStartE2EDuration="1.797939134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:15.79627107 +0000 UTC m=+1021.007464327" watchObservedRunningTime="2026-09-10 13:34:15.797939134 +0000 UTC m=+1021.009132390"
Sep 10 13:34:18.755091 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:18.755055    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:18.819826 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:18.819789    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:19.096031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.096001    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:19.425820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.425741    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"]
Sep 10 13:34:19.429161 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.429144    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"
Sep 10 13:34:19.436110 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.436086    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:34:19.436242 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.436226    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:34:19.447512 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.447488    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"]
Sep 10 13:34:19.556246 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.556205    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadad5d9-6169-4956-9a8b-182b42804e1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw\" (UID: \"eadad5d9-6169-4956-9a8b-182b42804e1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"
Sep 10 13:34:19.584951 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.584925    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:19.657594 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.657560    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadad5d9-6169-4956-9a8b-182b42804e1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw\" (UID: \"eadad5d9-6169-4956-9a8b-182b42804e1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"
Sep 10 13:34:19.658216 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.658196    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadad5d9-6169-4956-9a8b-182b42804e1b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw\" (UID: \"eadad5d9-6169-4956-9a8b-182b42804e1b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"
Sep 10 13:34:19.738625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:19.738540    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"
Sep 10 13:34:20.011979 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:20.011955    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"]
Sep 10 13:34:20.014140 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:34:20.014112    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeadad5d9_6169_4956_9a8b_182b42804e1b.slice/crio-41899608435d30ac5f101150399b833d1128c163258b4ea916e9973cdc7bd783 WatchSource:0}: Error finding container 41899608435d30ac5f101150399b833d1128c163258b4ea916e9973cdc7bd783: Status 404 returned error can't find the container with id 41899608435d30ac5f101150399b833d1128c163258b4ea916e9973cdc7bd783
Sep 10 13:34:20.780067 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:20.780039    2551 generic.go:356] "Generic (PLEG): container finished" podID="eadad5d9-6169-4956-9a8b-182b42804e1b" containerID="2c4a88627f9c88990a1272587c39b23f25e0a158832e7f3b22daccec22cd2bd5" exitCode=0
Sep 10 13:34:20.780192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:20.780083    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw" event={"ID":"eadad5d9-6169-4956-9a8b-182b42804e1b","Type":"ContainerDied","Data":"2c4a88627f9c88990a1272587c39b23f25e0a158832e7f3b22daccec22cd2bd5"}
Sep 10 13:34:20.780192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:20.780134    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw" event={"ID":"eadad5d9-6169-4956-9a8b-182b42804e1b","Type":"ContainerStarted","Data":"41899608435d30ac5f101150399b833d1128c163258b4ea916e9973cdc7bd783"}
Sep 10 13:34:21.908138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:21.908113    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"
Sep 10 13:34:22.081393 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:22.081311    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadad5d9-6169-4956-9a8b-182b42804e1b-maas-api-service-ca\") pod \"eadad5d9-6169-4956-9a8b-182b42804e1b\" (UID: \"eadad5d9-6169-4956-9a8b-182b42804e1b\") "
Sep 10 13:34:22.081655 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:22.081633    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eadad5d9-6169-4956-9a8b-182b42804e1b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eadad5d9-6169-4956-9a8b-182b42804e1b" (UID: "eadad5d9-6169-4956-9a8b-182b42804e1b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:34:22.182509 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:22.182482    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadad5d9-6169-4956-9a8b-182b42804e1b-maas-api-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:22.552938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:22.552899    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:22.790807 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:22.790779    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"
Sep 10 13:34:22.790807 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:22.790788    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw" event={"ID":"eadad5d9-6169-4956-9a8b-182b42804e1b","Type":"ContainerDied","Data":"41899608435d30ac5f101150399b833d1128c163258b4ea916e9973cdc7bd783"}
Sep 10 13:34:22.791008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:22.790830    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41899608435d30ac5f101150399b833d1128c163258b4ea916e9973cdc7bd783"
Sep 10 13:34:23.845991 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:23.845955    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:34:26.760706 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:26.760653    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:30.065353 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.065318    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:30.071021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.070992    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"]
Sep 10 13:34:30.073579 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.073557    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-6beee4-29817450-jwnb9"]
Sep 10 13:34:30.123318 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.123283    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"]
Sep 10 13:34:30.123540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.123512    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b" podUID="eca57a56-67bf-493e-aa29-b0a3c186b691" containerName="payload-processing" containerID="cri-o://2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364" gracePeriod=30
Sep 10 13:34:30.131764 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.131724    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"]
Sep 10 13:34:30.132183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.131989    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn" podUID="476d37a6-a48d-4af5-9f07-81df1c767d86" containerName="payload-pre-processing" containerID="cri-o://fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566" gracePeriod=30
Sep 10 13:34:30.173652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.173615    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:30.280942 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.280905    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:30.371276 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.371245    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:30.381931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.381898    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"]
Sep 10 13:34:30.384406 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.384379    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6beee4-4njpw"]
Sep 10 13:34:30.407113 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.407090    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:34:30.409917 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.409896    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:34:30.438953 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.438925    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:30.562815 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.562780    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zw62n\" (UniqueName: \"kubernetes.io/projected/476d37a6-a48d-4af5-9f07-81df1c767d86-kube-api-access-zw62n\") pod \"476d37a6-a48d-4af5-9f07-81df1c767d86\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") "
Sep 10 13:34:30.563009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.562836    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/476d37a6-a48d-4af5-9f07-81df1c767d86-plugins-config-volume\") pod \"476d37a6-a48d-4af5-9f07-81df1c767d86\" (UID: \"476d37a6-a48d-4af5-9f07-81df1c767d86\") "
Sep 10 13:34:30.563009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.562865    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca57a56-67bf-493e-aa29-b0a3c186b691-plugins-config-volume\") pod \"eca57a56-67bf-493e-aa29-b0a3c186b691\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") "
Sep 10 13:34:30.563009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.562882    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54bdz\" (UniqueName: \"kubernetes.io/projected/eca57a56-67bf-493e-aa29-b0a3c186b691-kube-api-access-54bdz\") pod \"eca57a56-67bf-493e-aa29-b0a3c186b691\" (UID: \"eca57a56-67bf-493e-aa29-b0a3c186b691\") "
Sep 10 13:34:30.563246 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.563220    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476d37a6-a48d-4af5-9f07-81df1c767d86-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "476d37a6-a48d-4af5-9f07-81df1c767d86" (UID: "476d37a6-a48d-4af5-9f07-81df1c767d86"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:34:30.563246 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.563237    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eca57a56-67bf-493e-aa29-b0a3c186b691-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eca57a56-67bf-493e-aa29-b0a3c186b691" (UID: "eca57a56-67bf-493e-aa29-b0a3c186b691"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:34:30.565012 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.564989    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/476d37a6-a48d-4af5-9f07-81df1c767d86-kube-api-access-zw62n" (OuterVolumeSpecName: "kube-api-access-zw62n") pod "476d37a6-a48d-4af5-9f07-81df1c767d86" (UID: "476d37a6-a48d-4af5-9f07-81df1c767d86"). InnerVolumeSpecName "kube-api-access-zw62n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:34:30.565123 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.565105    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eca57a56-67bf-493e-aa29-b0a3c186b691-kube-api-access-54bdz" (OuterVolumeSpecName: "kube-api-access-54bdz") pod "eca57a56-67bf-493e-aa29-b0a3c186b691" (UID: "eca57a56-67bf-493e-aa29-b0a3c186b691"). InnerVolumeSpecName "kube-api-access-54bdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:34:30.663699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.663591    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zw62n\" (UniqueName: \"kubernetes.io/projected/476d37a6-a48d-4af5-9f07-81df1c767d86-kube-api-access-zw62n\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:30.663699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.663625    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/476d37a6-a48d-4af5-9f07-81df1c767d86-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:30.663699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.663637    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eca57a56-67bf-493e-aa29-b0a3c186b691-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:30.663699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.663646    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54bdz\" (UniqueName: \"kubernetes.io/projected/eca57a56-67bf-493e-aa29-b0a3c186b691-kube-api-access-54bdz\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:30.793629 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.793586    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:30.830389 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.830355    2551 generic.go:356] "Generic (PLEG): container finished" podID="eca57a56-67bf-493e-aa29-b0a3c186b691" containerID="2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364" exitCode=0
Sep 10 13:34:30.830546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.830407    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"
Sep 10 13:34:30.830546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.830451    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b" event={"ID":"eca57a56-67bf-493e-aa29-b0a3c186b691","Type":"ContainerDied","Data":"2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364"}
Sep 10 13:34:30.830546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.830512    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b" event={"ID":"eca57a56-67bf-493e-aa29-b0a3c186b691","Type":"ContainerDied","Data":"2c74404e1485f336d6decc246289598cfe290f4d2bf0237f6f36b9463da794fb"}
Sep 10 13:34:30.830704 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.830549    2551 scope.go:117] "RemoveContainer" containerID="2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364"
Sep 10 13:34:30.832025 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.832003    2551 generic.go:356] "Generic (PLEG): container finished" podID="476d37a6-a48d-4af5-9f07-81df1c767d86" containerID="fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566" exitCode=0
Sep 10 13:34:30.832150 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.832048    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"
Sep 10 13:34:30.832150 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.832063    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn" event={"ID":"476d37a6-a48d-4af5-9f07-81df1c767d86","Type":"ContainerDied","Data":"fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566"}
Sep 10 13:34:30.832150 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.832097    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn" event={"ID":"476d37a6-a48d-4af5-9f07-81df1c767d86","Type":"ContainerDied","Data":"beb265d5f71c6ed9c9ea69846475c15c645f3dc3cf2387330e4ddb39edd1ba15"}
Sep 10 13:34:30.839812 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.839792    2551 scope.go:117] "RemoveContainer" containerID="2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364"
Sep 10 13:34:30.840141 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:34:30.840119    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364\": container with ID starting with 2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364 not found: ID does not exist" containerID="2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364"
Sep 10 13:34:30.840243 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.840151    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364"} err="failed to get container status \"2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364\": rpc error: code = NotFound desc = could not find container \"2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364\": container with ID starting with 2c69676733079506dbc1b28da76a5c35dfbf2e4bb6644bc19c5f893d2e7c2364 not found: ID does not exist"
Sep 10 13:34:30.840243 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.840174    2551 scope.go:117] "RemoveContainer" containerID="fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566"
Sep 10 13:34:30.848279 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.848260    2551 scope.go:117] "RemoveContainer" containerID="fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566"
Sep 10 13:34:30.848548 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:34:30.848533    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566\": container with ID starting with fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566 not found: ID does not exist" containerID="fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566"
Sep 10 13:34:30.848590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.848554    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566"} err="failed to get container status \"fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566\": rpc error: code = NotFound desc = could not find container \"fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566\": container with ID starting with fb73717d7b5d74ab148f1b476c609e752d32b2e6da31b4da4b37482f061e1566 not found: ID does not exist"
Sep 10 13:34:30.854773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.854751    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"]
Sep 10 13:34:30.858581 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.858557    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6beee4-969d6bc97-wgd8b"]
Sep 10 13:34:30.872072 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.872045    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"]
Sep 10 13:34:30.876583 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:30.876557    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6beee4-79d9fcfb4b-pmxtn"]
Sep 10 13:34:31.249509 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:31.249474    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="476d37a6-a48d-4af5-9f07-81df1c767d86" path="/var/lib/kubelet/pods/476d37a6-a48d-4af5-9f07-81df1c767d86/volumes"
Sep 10 13:34:31.249897 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:31.249777    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54a66261-58f8-4cb0-bf15-2ebe961e11a5" path="/var/lib/kubelet/pods/54a66261-58f8-4cb0-bf15-2ebe961e11a5/volumes"
Sep 10 13:34:31.250022 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:31.250010    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eadad5d9-6169-4956-9a8b-182b42804e1b" path="/var/lib/kubelet/pods/eadad5d9-6169-4956-9a8b-182b42804e1b/volumes"
Sep 10 13:34:31.250322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:31.250310    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eca57a56-67bf-493e-aa29-b0a3c186b691" path="/var/lib/kubelet/pods/eca57a56-67bf-493e-aa29-b0a3c186b691/volumes"
Sep 10 13:34:34.497605 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:34.497567    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:34:35.468076 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:35.468038    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:37.388377 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:37.388341    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:37.488538 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:37.488505    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:37.918478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:37.918442    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:38.196586 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:38.196496    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:38.402941 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:38.402905    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:38.800612 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:38.800577    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:39.010231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:39.010201    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:39.106027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:39.105923    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:39.404757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:39.404649    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:39.504849 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:39.504818    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:42.301143 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:42.301111    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:42.399192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:42.399160    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:43.496185 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:43.496159    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:43.590809 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:43.590782    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:43.690064 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:43.690039    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:44.041958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.041924    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:44.140065 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.140035    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:44.181490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181463    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"]
Sep 10 13:34:44.181834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181810    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eca57a56-67bf-493e-aa29-b0a3c186b691" containerName="payload-processing"
Sep 10 13:34:44.181834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181828    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="eca57a56-67bf-493e-aa29-b0a3c186b691" containerName="payload-processing"
Sep 10 13:34:44.181834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181838    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eadad5d9-6169-4956-9a8b-182b42804e1b" containerName="revoke-keys"
Sep 10 13:34:44.182009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181845    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="eadad5d9-6169-4956-9a8b-182b42804e1b" containerName="revoke-keys"
Sep 10 13:34:44.182009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181858    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476d37a6-a48d-4af5-9f07-81df1c767d86" containerName="payload-pre-processing"
Sep 10 13:34:44.182009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181863    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="476d37a6-a48d-4af5-9f07-81df1c767d86" containerName="payload-pre-processing"
Sep 10 13:34:44.182009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181919    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="476d37a6-a48d-4af5-9f07-81df1c767d86" containerName="payload-pre-processing"
Sep 10 13:34:44.182009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181926    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="eadad5d9-6169-4956-9a8b-182b42804e1b" containerName="revoke-keys"
Sep 10 13:34:44.182009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.181932    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="eca57a56-67bf-493e-aa29-b0a3c186b691" containerName="payload-processing"
Sep 10 13:34:44.184837 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.184817    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.188273 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.188250    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-7657ef\""
Sep 10 13:34:44.188435 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.188417    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-7657ef-dockercfg-r8cpg\""
Sep 10 13:34:44.201096 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.201071    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"]
Sep 10 13:34:44.214547 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.214522    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"]
Sep 10 13:34:44.217823 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.217806    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.235210 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.235189    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"]
Sep 10 13:34:44.278868 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.278839    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsz4h\" (UniqueName: \"kubernetes.io/projected/cfcfd673-94f5-4601-bb1e-e562075c207f-kube-api-access-gsz4h\") pod \"payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.278996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.278886    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfcfd673-94f5-4601-bb1e-e562075c207f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.279049 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.279024    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65f92dec-a0b2-467b-9b90-c9b218ff556d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.279089 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.279071    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp9gd\" (UniqueName: \"kubernetes.io/projected/65f92dec-a0b2-467b-9b90-c9b218ff556d-kube-api-access-wp9gd\") pod \"payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.380299 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.380217    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsz4h\" (UniqueName: \"kubernetes.io/projected/cfcfd673-94f5-4601-bb1e-e562075c207f-kube-api-access-gsz4h\") pod \"payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.380299 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.380267    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfcfd673-94f5-4601-bb1e-e562075c207f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.380505 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.380322    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65f92dec-a0b2-467b-9b90-c9b218ff556d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.380505 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.380349    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp9gd\" (UniqueName: \"kubernetes.io/projected/65f92dec-a0b2-467b-9b90-c9b218ff556d-kube-api-access-wp9gd\") pod \"payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.381005 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.380986    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfcfd673-94f5-4601-bb1e-e562075c207f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.381071 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.381052    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65f92dec-a0b2-467b-9b90-c9b218ff556d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.389724 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.389699    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp9gd\" (UniqueName: \"kubernetes.io/projected/65f92dec-a0b2-467b-9b90-c9b218ff556d-kube-api-access-wp9gd\") pod \"payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.389822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.389805    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsz4h\" (UniqueName: \"kubernetes.io/projected/cfcfd673-94f5-4601-bb1e-e562075c207f-kube-api-access-gsz4h\") pod \"payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.495266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.495236    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.527592 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.527566    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.790333 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:34:44.790302    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfcfd673_94f5_4601_bb1e_e562075c207f.slice/crio-a0e0c99c4d6d89687a06d4a5719756685c0e06f7a6e1def860223609b770c960 WatchSource:0}: Error finding container a0e0c99c4d6d89687a06d4a5719756685c0e06f7a6e1def860223609b770c960: Status 404 returned error can't find the container with id a0e0c99c4d6d89687a06d4a5719756685c0e06f7a6e1def860223609b770c960
Sep 10 13:34:44.791308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.791279    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"]
Sep 10 13:34:44.833696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.833673    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"]
Sep 10 13:34:44.836779 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:34:44.836752    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f92dec_a0b2_467b_9b90_c9b218ff556d.slice/crio-5d548fe67488866e107d8a6e0d62332b7cbabe9d7743640a2ecc95638b3bb515 WatchSource:0}: Error finding container 5d548fe67488866e107d8a6e0d62332b7cbabe9d7743640a2ecc95638b3bb515: Status 404 returned error can't find the container with id 5d548fe67488866e107d8a6e0d62332b7cbabe9d7743640a2ecc95638b3bb515
Sep 10 13:34:44.908258 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.908230    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d" event={"ID":"65f92dec-a0b2-467b-9b90-c9b218ff556d","Type":"ContainerStarted","Data":"41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc"}
Sep 10 13:34:44.908391 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.908296    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:44.908391 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.908322    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d" event={"ID":"65f92dec-a0b2-467b-9b90-c9b218ff556d","Type":"ContainerStarted","Data":"5d548fe67488866e107d8a6e0d62332b7cbabe9d7743640a2ecc95638b3bb515"}
Sep 10 13:34:44.909990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.909963    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv" event={"ID":"cfcfd673-94f5-4601-bb1e-e562075c207f","Type":"ContainerStarted","Data":"7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6"}
Sep 10 13:34:44.910101 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.910015    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv" event={"ID":"cfcfd673-94f5-4601-bb1e-e562075c207f","Type":"ContainerStarted","Data":"a0e0c99c4d6d89687a06d4a5719756685c0e06f7a6e1def860223609b770c960"}
Sep 10 13:34:44.910101 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.910065    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:44.975287 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:44.975227    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d" podStartSLOduration=0.975206382 podStartE2EDuration="975.206382ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:44.961277189 +0000 UTC m=+1050.172470451" watchObservedRunningTime="2026-09-10 13:34:44.975206382 +0000 UTC m=+1050.186399643"
Sep 10 13:34:45.023615 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:45.023559    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv" podStartSLOduration=1.023539612 podStartE2EDuration="1.023539612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:34:45.022696081 +0000 UTC m=+1050.233889336" watchObservedRunningTime="2026-09-10 13:34:45.023539612 +0000 UTC m=+1050.234732871"
Sep 10 13:34:46.344180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:46.344137    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:46.468628 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:46.468601    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:47.043540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:47.043507    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:47.793370 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:47.793342    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:34:55.916250 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:55.916217    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:34:55.916681 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:55.916292    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:34:57.823089 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:57.823040    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:57.969102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:57.969066    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"]
Sep 10 13:34:57.969292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:57.969270    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v" podUID="beeb1520-a721-44c7-802d-aa2d7a00d074" containerName="payload-pre-processing" containerID="cri-o://0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2" gracePeriod=30
Sep 10 13:34:58.010406 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.010377    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:58.154883 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.154848    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:58.218541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.218517    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:58.303541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.303507    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beeb1520-a721-44c7-802d-aa2d7a00d074-plugins-config-volume\") pod \"beeb1520-a721-44c7-802d-aa2d7a00d074\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") "
Sep 10 13:34:58.303541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.303546    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbbz8\" (UniqueName: \"kubernetes.io/projected/beeb1520-a721-44c7-802d-aa2d7a00d074-kube-api-access-rbbz8\") pod \"beeb1520-a721-44c7-802d-aa2d7a00d074\" (UID: \"beeb1520-a721-44c7-802d-aa2d7a00d074\") "
Sep 10 13:34:58.303964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.303941    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/beeb1520-a721-44c7-802d-aa2d7a00d074-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "beeb1520-a721-44c7-802d-aa2d7a00d074" (UID: "beeb1520-a721-44c7-802d-aa2d7a00d074"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:34:58.305641 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.305618    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/beeb1520-a721-44c7-802d-aa2d7a00d074-kube-api-access-rbbz8" (OuterVolumeSpecName: "kube-api-access-rbbz8") pod "beeb1520-a721-44c7-802d-aa2d7a00d074" (UID: "beeb1520-a721-44c7-802d-aa2d7a00d074"). InnerVolumeSpecName "kube-api-access-rbbz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:34:58.342757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.342657    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:58.405057 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.405028    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/beeb1520-a721-44c7-802d-aa2d7a00d074-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:58.405057 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.405055    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbbz8\" (UniqueName: \"kubernetes.io/projected/beeb1520-a721-44c7-802d-aa2d7a00d074-kube-api-access-rbbz8\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:34:58.438987 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.438956    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:58.875984 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.875949    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:34:58.982940 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.982905    2551 generic.go:356] "Generic (PLEG): container finished" podID="beeb1520-a721-44c7-802d-aa2d7a00d074" containerID="0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2" exitCode=0
Sep 10 13:34:58.983093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.982944    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v" event={"ID":"beeb1520-a721-44c7-802d-aa2d7a00d074","Type":"ContainerDied","Data":"0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2"}
Sep 10 13:34:58.983093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.983004    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v" event={"ID":"beeb1520-a721-44c7-802d-aa2d7a00d074","Type":"ContainerDied","Data":"49e0a101aeec0a5b0bb6a31f7bb7bedf001fb66beba9bf43e64c94b7675e77a4"}
Sep 10 13:34:58.983093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.983027    2551 scope.go:117] "RemoveContainer" containerID="0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2"
Sep 10 13:34:58.983093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.982955    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"
Sep 10 13:34:58.991869 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.991848    2551 scope.go:117] "RemoveContainer" containerID="0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2"
Sep 10 13:34:58.992165 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:34:58.992147    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2\": container with ID starting with 0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2 not found: ID does not exist" containerID="0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2"
Sep 10 13:34:58.992213 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:58.992175    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2"} err="failed to get container status \"0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2\": rpc error: code = NotFound desc = could not find container \"0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2\": container with ID starting with 0707dbdc99ebb9c0113145cd4e47c578becb80ce10cddca4137b897693f976b2 not found: ID does not exist"
Sep 10 13:34:59.020603 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:59.020574    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"]
Sep 10 13:34:59.025435 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:59.025408    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7f8babf4-7c6cb599d-7rx7v"]
Sep 10 13:34:59.249077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:34:59.249001    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="beeb1520-a721-44c7-802d-aa2d7a00d074" path="/var/lib/kubelet/pods/beeb1520-a721-44c7-802d-aa2d7a00d074/volumes"
Sep 10 13:35:01.214413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:01.214376    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:35:04.736758 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:04.736715    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:07.449853 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:07.449812    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:07.818012 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:07.817911    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:07.913043 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:07.913016    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:08.117824 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:08.117737    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:08.324005 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:08.323928    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:08.549906 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:08.549871    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:08.645021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:08.644992    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:08.770333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:08.770299    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:08.960787 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:08.960675    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:11.747908 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:11.747872    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:11.905648 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:11.905602    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:11.995787 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:11.995750    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:13.069388 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:13.069359    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:13.161848 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:13.161800    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:13.782613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:13.782579    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:13.981543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:13.981508    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:14.110707 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:14.110595    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:16.722066 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:16.722029    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:16.786492 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:16.786459    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:16.911613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:16.911580    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:16.996233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:16.996154    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:17.206064 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:17.206021    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:17.858469 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:17.858436    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:35:24.245345 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:24.245315    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:24.339239 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:24.339212    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:25.283777 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:25.283741    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:25.716973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:25.716897    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:25.933300 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:25.933266    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:28.607830 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:28.607798    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:28.695872 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:28.695840    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:29.815395 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:29.815363    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:29.912214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:29.912175    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:30.014549 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.014521    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:30.498452 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.498418    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:30.594137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.594105    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:30.702566 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.702540    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"]
Sep 10 13:35:30.702904 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.702886    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="beeb1520-a721-44c7-802d-aa2d7a00d074" containerName="payload-pre-processing"
Sep 10 13:35:30.703007 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.702906    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="beeb1520-a721-44c7-802d-aa2d7a00d074" containerName="payload-pre-processing"
Sep 10 13:35:30.703066 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.703021    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="beeb1520-a721-44c7-802d-aa2d7a00d074" containerName="payload-pre-processing"
Sep 10 13:35:30.707408 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.707390    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:30.710468 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.710450    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-c165a4\""
Sep 10 13:35:30.710555 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.710479    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-c165a4-dockercfg-zm7pv\""
Sep 10 13:35:30.722651 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.722620    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"]
Sep 10 13:35:30.763920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.763869    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"]
Sep 10 13:35:30.766913 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.766895    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:30.781806 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.781778    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfn8d\" (UniqueName: \"kubernetes.io/projected/ad92952b-9156-4ac8-bd9b-01797d5e03f3-kube-api-access-gfn8d\") pod \"payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:30.781904 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.781884    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f973059-047f-40a7-83a3-f1842d5d20ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:30.781971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.781915    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad92952b-9156-4ac8-bd9b-01797d5e03f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:30.781971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.781960    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d6gf\" (UniqueName: \"kubernetes.io/projected/3f973059-047f-40a7-83a3-f1842d5d20ed-kube-api-access-2d6gf\") pod \"payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:30.786325 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.786303    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"]
Sep 10 13:35:30.883305 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.883270    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f973059-047f-40a7-83a3-f1842d5d20ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:30.883305 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.883310    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad92952b-9156-4ac8-bd9b-01797d5e03f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:30.883741 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.883329    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d6gf\" (UniqueName: \"kubernetes.io/projected/3f973059-047f-40a7-83a3-f1842d5d20ed-kube-api-access-2d6gf\") pod \"payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:30.883741 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.883367    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfn8d\" (UniqueName: \"kubernetes.io/projected/ad92952b-9156-4ac8-bd9b-01797d5e03f3-kube-api-access-gfn8d\") pod \"payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:30.884285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.884261    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad92952b-9156-4ac8-bd9b-01797d5e03f3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:30.884322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.884261    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f973059-047f-40a7-83a3-f1842d5d20ed-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:30.891737 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.891715    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfn8d\" (UniqueName: \"kubernetes.io/projected/ad92952b-9156-4ac8-bd9b-01797d5e03f3-kube-api-access-gfn8d\") pod \"payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:30.891995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:30.891975    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d6gf\" (UniqueName: \"kubernetes.io/projected/3f973059-047f-40a7-83a3-f1842d5d20ed-kube-api-access-2d6gf\") pod \"payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:31.017414 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:31.017337    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:31.076137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:31.076105    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:31.293614 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:31.293590    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"]
Sep 10 13:35:31.295702 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:35:31.295657    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad92952b_9156_4ac8_bd9b_01797d5e03f3.slice/crio-f4d5a12a69fd88fdf0eaac7c980642d9f5ab835d2c816e669c4371ee6a772470 WatchSource:0}: Error finding container f4d5a12a69fd88fdf0eaac7c980642d9f5ab835d2c816e669c4371ee6a772470: Status 404 returned error can't find the container with id f4d5a12a69fd88fdf0eaac7c980642d9f5ab835d2c816e669c4371ee6a772470
Sep 10 13:35:31.366305 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:31.366280    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"]
Sep 10 13:35:31.367489 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:35:31.367468    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f973059_047f_40a7_83a3_f1842d5d20ed.slice/crio-145cda0dc31620ee4d7357d4c9e16b3edd518b1ac545f3ebd49c588e04ecfbab WatchSource:0}: Error finding container 145cda0dc31620ee4d7357d4c9e16b3edd518b1ac545f3ebd49c588e04ecfbab: Status 404 returned error can't find the container with id 145cda0dc31620ee4d7357d4c9e16b3edd518b1ac545f3ebd49c588e04ecfbab
Sep 10 13:35:32.147528 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.147491    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm" event={"ID":"3f973059-047f-40a7-83a3-f1842d5d20ed","Type":"ContainerStarted","Data":"884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4"}
Sep 10 13:35:32.148023 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.147553    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm" event={"ID":"3f973059-047f-40a7-83a3-f1842d5d20ed","Type":"ContainerStarted","Data":"145cda0dc31620ee4d7357d4c9e16b3edd518b1ac545f3ebd49c588e04ecfbab"}
Sep 10 13:35:32.148023 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.147601    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:32.149033 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.149013    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8" event={"ID":"ad92952b-9156-4ac8-bd9b-01797d5e03f3","Type":"ContainerStarted","Data":"d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f"}
Sep 10 13:35:32.149112 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.149045    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8" event={"ID":"ad92952b-9156-4ac8-bd9b-01797d5e03f3","Type":"ContainerStarted","Data":"f4d5a12a69fd88fdf0eaac7c980642d9f5ab835d2c816e669c4371ee6a772470"}
Sep 10 13:35:32.149160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.149141    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:32.172339 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.172278    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm" podStartSLOduration=2.172259799 podStartE2EDuration="2.172259799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:32.172227016 +0000 UTC m=+1097.383420275" watchObservedRunningTime="2026-09-10 13:35:32.172259799 +0000 UTC m=+1097.383453060"
Sep 10 13:35:32.193803 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:32.193755    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8" podStartSLOduration=2.193737338 podStartE2EDuration="2.193737338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:32.192635648 +0000 UTC m=+1097.403828908" watchObservedRunningTime="2026-09-10 13:35:32.193737338 +0000 UTC m=+1097.404930598"
Sep 10 13:35:34.300197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:34.300161    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:35:36.168983 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:36.168950    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:41.827262 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:41.827227    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:43.156320 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:43.156286    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:35:43.156655 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:43.156357    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:35:45.329862 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.329825    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"]
Sep 10 13:35:45.333457 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.333438    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.336574 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.336550    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-7657ef\"/\"kube-root-ca.crt\""
Sep 10 13:35:45.336738 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.336551    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7657ef\"/\"auth-test-model-7657ef-kserve-self-signed-certs\""
Sep 10 13:35:45.337660 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.337638    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-7657ef\"/\"openshift-service-ca.crt\""
Sep 10 13:35:45.337889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.337870    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7657ef\"/\"default-dockercfg-prfph\""
Sep 10 13:35:45.345041 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.345021    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"]
Sep 10 13:35:45.356264 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.356242    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:45.395077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.395040    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-model-cache\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.395196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.395089    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tls-certs\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.395196 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.395160    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjbgn\" (UniqueName: \"kubernetes.io/projected/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-kube-api-access-sjbgn\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.395295 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.395210    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-home\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.395295 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.395250    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tmp-dir\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.395295 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.395271    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-dshm\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.496595 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.496556    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjbgn\" (UniqueName: \"kubernetes.io/projected/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-kube-api-access-sjbgn\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.496788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.496628    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-home\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.496788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.496698    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tmp-dir\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.496788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.496729    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-dshm\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.496973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.496789    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-model-cache\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.496973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.496817    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tls-certs\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.497044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.497017    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-home\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.497102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.497079    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tmp-dir\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.497142 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.497132    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-model-cache\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.499044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.499015    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-dshm\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.499618 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.499599    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tls-certs\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.505622 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.505598    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjbgn\" (UniqueName: \"kubernetes.io/projected/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-kube-api-access-sjbgn\") pod \"auth-test-model-7657ef-kserve-76596fd9bc-frdvq\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.643986 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.643918    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:45.919972 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:45.919920    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"]
Sep 10 13:35:45.922014 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:35:45.921984    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod599996d1_3cdb_431d_b6ad_dd1ba1d5e999.slice/crio-7bb43a9582866c4c84f10d3ea1f499d2e522c5b370d9942ff246571f2dfc6943 WatchSource:0}: Error finding container 7bb43a9582866c4c84f10d3ea1f499d2e522c5b370d9942ff246571f2dfc6943: Status 404 returned error can't find the container with id 7bb43a9582866c4c84f10d3ea1f499d2e522c5b370d9942ff246571f2dfc6943
Sep 10 13:35:46.216777 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:46.216696    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq" event={"ID":"599996d1-3cdb-431d-b6ad-dd1ba1d5e999","Type":"ContainerStarted","Data":"f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46"}
Sep 10 13:35:46.216777 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:46.216748    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq" event={"ID":"599996d1-3cdb-431d-b6ad-dd1ba1d5e999","Type":"ContainerStarted","Data":"7bb43a9582866c4c84f10d3ea1f499d2e522c5b370d9942ff246571f2dfc6943"}
Sep 10 13:35:46.242301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:46.242248    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq" podStartSLOduration=1.242233469 podStartE2EDuration="1.242233469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:35:46.239995756 +0000 UTC m=+1111.451189014" watchObservedRunningTime="2026-09-10 13:35:46.242233469 +0000 UTC m=+1111.453426726"
Sep 10 13:35:49.100159 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:49.100125    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:35:51.122858 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:51.122824    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:53.680590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:53.680557    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:54.031541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:54.031513    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:54.398531 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:54.398453    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:54.921443 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:54.921413    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:55.644598 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:55.644559    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:55.644785 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:55.644614    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:55.657645 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:55.657614    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:56.278323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:56.278297    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:35:57.718098 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:57.718066    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:57.811233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:57.811205    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:58.908106 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:58.908078    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:59.007417 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.007389    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:59.109323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.109295    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:59.641852 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.641822    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:59.822796 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.822764    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:35:59.896226 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.896156    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"]
Sep 10 13:35:59.899646 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.899629    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:35:59.903674 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.903638    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-e65037-dockercfg-fnh7w\""
Sep 10 13:35:59.903788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.903726    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-e65037\""
Sep 10 13:35:59.913548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:35:59.913521    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"]
Sep 10 13:36:00.022289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.022256    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02897304-5217-4697-8e96-9d0cf4ec83c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:00.022437 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.022315    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8455z\" (UniqueName: \"kubernetes.io/projected/02897304-5217-4697-8e96-9d0cf4ec83c1-kube-api-access-8455z\") pod \"payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:00.122882 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.122845    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8455z\" (UniqueName: \"kubernetes.io/projected/02897304-5217-4697-8e96-9d0cf4ec83c1-kube-api-access-8455z\") pod \"payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:00.123003 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.122954    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02897304-5217-4697-8e96-9d0cf4ec83c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:00.123708 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.123689    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02897304-5217-4697-8e96-9d0cf4ec83c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:00.142501 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.142472    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8455z\" (UniqueName: \"kubernetes.io/projected/02897304-5217-4697-8e96-9d0cf4ec83c1-kube-api-access-8455z\") pod \"payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:00.210436 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.210372    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:00.485873 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:00.485845    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"]
Sep 10 13:36:00.486827 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:36:00.486801    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02897304_5217_4697_8e96_9d0cf4ec83c1.slice/crio-8eeee99143b267adaf83d229069a742fb31274b4b6d350bbd4cf413989f2be1a WatchSource:0}: Error finding container 8eeee99143b267adaf83d229069a742fb31274b4b6d350bbd4cf413989f2be1a: Status 404 returned error can't find the container with id 8eeee99143b267adaf83d229069a742fb31274b4b6d350bbd4cf413989f2be1a
Sep 10 13:36:01.301826 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:01.301788    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb" event={"ID":"02897304-5217-4697-8e96-9d0cf4ec83c1","Type":"ContainerStarted","Data":"bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9"}
Sep 10 13:36:01.302218 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:01.301848    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb" event={"ID":"02897304-5217-4697-8e96-9d0cf4ec83c1","Type":"ContainerStarted","Data":"8eeee99143b267adaf83d229069a742fb31274b4b6d350bbd4cf413989f2be1a"}
Sep 10 13:36:01.302218 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:01.301933    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:06.590317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:06.590249    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb" podStartSLOduration=7.590232196 podStartE2EDuration="7.590232196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:01.329887254 +0000 UTC m=+1126.541080510" watchObservedRunningTime="2026-09-10 13:36:06.590232196 +0000 UTC m=+1131.801425471"
Sep 10 13:36:06.591070 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:06.591052    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:36:12.308748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.308719    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:36:12.633964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.633894    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"]
Sep 10 13:36:12.638387 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.638369    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.641693 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.641656    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"default-dockercfg-g5jcc\""
Sep 10 13:36:12.641693 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.641685    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"openshift-service-ca.crt\""
Sep 10 13:36:12.643149 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.643128    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"kube-root-ca.crt\""
Sep 10 13:36:12.643246 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.643155    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"auth-test-model-7739cb-kserve-self-signed-certs\""
Sep 10 13:36:12.648819 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.648798    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"]
Sep 10 13:36:12.660088 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.660070    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:12.724815 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.724789    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tls-certs\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.724921 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.724824    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-home\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.724921 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.724849    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-model-cache\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.724921 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.724875    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw25b\" (UniqueName: \"kubernetes.io/projected/9217f8be-4e16-4dea-96d9-e46e7c21fe23-kube-api-access-kw25b\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.725080 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.724922    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-dshm\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.725080 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.724939    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tmp-dir\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.825886 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.825860    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-model-cache\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.825994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.825899    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw25b\" (UniqueName: \"kubernetes.io/projected/9217f8be-4e16-4dea-96d9-e46e7c21fe23-kube-api-access-kw25b\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.825994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.825933    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-dshm\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.825994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.825950    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tmp-dir\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.826155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.826006    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tls-certs\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.826155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.826035    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-home\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.826323 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.826299    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-model-cache\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.826397 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.826361    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-home\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.826444 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.826393    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tmp-dir\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.828209 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.828189    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-dshm\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.828339 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.828321    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tls-certs\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.836945 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.836920    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw25b\" (UniqueName: \"kubernetes.io/projected/9217f8be-4e16-4dea-96d9-e46e7c21fe23-kube-api-access-kw25b\") pod \"auth-test-model-7739cb-kserve-6698796879-xpqbs\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:12.948448 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:12.948393    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:13.224195 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:13.224124    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"]
Sep 10 13:36:13.228439 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:36:13.228407    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9217f8be_4e16_4dea_96d9_e46e7c21fe23.slice/crio-513439a5986cafc563a4f1f3d142966fd9d3a19e845086bc37611cf0584e69e5 WatchSource:0}: Error finding container 513439a5986cafc563a4f1f3d142966fd9d3a19e845086bc37611cf0584e69e5: Status 404 returned error can't find the container with id 513439a5986cafc563a4f1f3d142966fd9d3a19e845086bc37611cf0584e69e5
Sep 10 13:36:13.358047 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:13.358013    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs" event={"ID":"9217f8be-4e16-4dea-96d9-e46e7c21fe23","Type":"ContainerStarted","Data":"fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2"}
Sep 10 13:36:13.358399 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:13.358055    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs" event={"ID":"9217f8be-4e16-4dea-96d9-e46e7c21fe23","Type":"ContainerStarted","Data":"513439a5986cafc563a4f1f3d142966fd9d3a19e845086bc37611cf0584e69e5"}
Sep 10 13:36:13.382425 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:13.382379    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs" podStartSLOduration=1.382363501 podStartE2EDuration="1.382363501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:13.381294272 +0000 UTC m=+1138.592487529" watchObservedRunningTime="2026-09-10 13:36:13.382363501 +0000 UTC m=+1138.593556751"
Sep 10 13:36:15.318338 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:15.318312    2551 scope.go:117] "RemoveContainer" containerID="aa920932ae73b26f9149f4b5d818b9826c89231210f46c45367b741ae041d285"
Sep 10 13:36:15.326343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:15.326322    2551 scope.go:117] "RemoveContainer" containerID="15cd7c2e0f4ce2d645514578208936ec9fffdcc6c05cc1c41116f26f305230d3"
Sep 10 13:36:15.333555 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:15.333527    2551 scope.go:117] "RemoveContainer" containerID="b8a231706b52e526932a8340d238ec0151fceb1e9fa93907de87041247a0ad6a"
Sep 10 13:36:17.692449 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:17.692400    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:36:22.789772 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:22.789741    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:22.949223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:22.949191    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:22.949401 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:22.949264    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:22.961502 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:22.961479    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:23.415686 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.415634    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:36:23.649509 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.649477    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7833-1b32e0"]
Sep 10 13:36:23.654491 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.654475    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:23.657240 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.657220    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tqk99\""
Sep 10 13:36:23.661410 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.661388    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7833-1b32e0"]
Sep 10 13:36:23.716719 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.716636    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvklm\" (UniqueName: \"kubernetes.io/projected/54c9b80c-5741-4a0f-926e-ead29e8addaf-kube-api-access-fvklm\") pod \"test-curl-7833-1b32e0\" (UID: \"54c9b80c-5741-4a0f-926e-ead29e8addaf\") " pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:23.817213 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.817183    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvklm\" (UniqueName: \"kubernetes.io/projected/54c9b80c-5741-4a0f-926e-ead29e8addaf-kube-api-access-fvklm\") pod \"test-curl-7833-1b32e0\" (UID: \"54c9b80c-5741-4a0f-926e-ead29e8addaf\") " pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:23.825834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.825808    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvklm\" (UniqueName: \"kubernetes.io/projected/54c9b80c-5741-4a0f-926e-ead29e8addaf-kube-api-access-fvklm\") pod \"test-curl-7833-1b32e0\" (UID: \"54c9b80c-5741-4a0f-926e-ead29e8addaf\") " pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:23.964992 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:23.964957    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:24.252626 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:24.252573    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7833-1b32e0"]
Sep 10 13:36:24.254978 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:36:24.254953    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54c9b80c_5741_4a0f_926e_ead29e8addaf.slice/crio-6f04b67654ac100e0675109e2d381272e938f53ea1d469eaf30e7cbee70a6a44 WatchSource:0}: Error finding container 6f04b67654ac100e0675109e2d381272e938f53ea1d469eaf30e7cbee70a6a44: Status 404 returned error can't find the container with id 6f04b67654ac100e0675109e2d381272e938f53ea1d469eaf30e7cbee70a6a44
Sep 10 13:36:24.410744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:24.410705    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7833-1b32e0" event={"ID":"54c9b80c-5741-4a0f-926e-ead29e8addaf","Type":"ContainerStarted","Data":"8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717"}
Sep 10 13:36:24.410892 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:24.410758    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7833-1b32e0" event={"ID":"54c9b80c-5741-4a0f-926e-ead29e8addaf","Type":"ContainerStarted","Data":"6f04b67654ac100e0675109e2d381272e938f53ea1d469eaf30e7cbee70a6a44"}
Sep 10 13:36:24.435143 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:24.435101    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7833-1b32e0" podStartSLOduration=1.43508715 podStartE2EDuration="1.43508715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:24.433364015 +0000 UTC m=+1149.644557271" watchObservedRunningTime="2026-09-10 13:36:24.43508715 +0000 UTC m=+1149.646280406"
Sep 10 13:36:25.106289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.106243    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7833-1b32e0"]
Sep 10 13:36:25.112228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.112206    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7833-1b32e0"]
Sep 10 13:36:25.237682 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.237639    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7833-5866b3"]
Sep 10 13:36:25.238025 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.238011    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" containerName="test-curl-7833-1b32e0"
Sep 10 13:36:25.238068 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.238027    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" containerName="test-curl-7833-1b32e0"
Sep 10 13:36:25.238107 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.238100    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" containerName="test-curl-7833-1b32e0"
Sep 10 13:36:25.241137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.241121    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-5866b3"
Sep 10 13:36:25.253293 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.253271    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7833-5866b3"]
Sep 10 13:36:25.331638 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.331612    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm57x\" (UniqueName: \"kubernetes.io/projected/bb3e3a2c-4325-475a-808e-c6ec67fa1bce-kube-api-access-cm57x\") pod \"test-curl-7833-5866b3\" (UID: \"bb3e3a2c-4325-475a-808e-c6ec67fa1bce\") " pod="opendatahub/test-curl-7833-5866b3"
Sep 10 13:36:25.415537 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:25.415471    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7833-1b32e0" container="test-curl-7833-1b32e0"
Sep 10 13:36:25.417953 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:25.417930    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:25.432931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.432907    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm57x\" (UniqueName: \"kubernetes.io/projected/bb3e3a2c-4325-475a-808e-c6ec67fa1bce-kube-api-access-cm57x\") pod \"test-curl-7833-5866b3\" (UID: \"bb3e3a2c-4325-475a-808e-c6ec67fa1bce\") " pod="opendatahub/test-curl-7833-5866b3"
Sep 10 13:36:25.442066 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.442039    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm57x\" (UniqueName: \"kubernetes.io/projected/bb3e3a2c-4325-475a-808e-c6ec67fa1bce-kube-api-access-cm57x\") pod \"test-curl-7833-5866b3\" (UID: \"bb3e3a2c-4325-475a-808e-c6ec67fa1bce\") " pod="opendatahub/test-curl-7833-5866b3"
Sep 10 13:36:25.549892 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.549869    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-5866b3"
Sep 10 13:36:25.818007 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:25.817944    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7833-5866b3"]
Sep 10 13:36:25.820077 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:36:25.820053    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb3e3a2c_4325_475a_808e_c6ec67fa1bce.slice/crio-52099bd72a2efedc63aff3ef7b1d24677431b6e1f180b8b55c697dfdd19e15fc WatchSource:0}: Error finding container 52099bd72a2efedc63aff3ef7b1d24677431b6e1f180b8b55c697dfdd19e15fc: Status 404 returned error can't find the container with id 52099bd72a2efedc63aff3ef7b1d24677431b6e1f180b8b55c697dfdd19e15fc
Sep 10 13:36:26.421168 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:26.421138    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7833-5866b3" event={"ID":"bb3e3a2c-4325-475a-808e-c6ec67fa1bce","Type":"ContainerStarted","Data":"2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7"}
Sep 10 13:36:26.421619 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:26.421179    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7833-5866b3" event={"ID":"bb3e3a2c-4325-475a-808e-c6ec67fa1bce","Type":"ContainerStarted","Data":"52099bd72a2efedc63aff3ef7b1d24677431b6e1f180b8b55c697dfdd19e15fc"}
Sep 10 13:36:26.421619 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:26.421181    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7833-1b32e0" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" containerName="test-curl-7833-1b32e0" containerID="cri-o://8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717" gracePeriod=2
Sep 10 13:36:26.423877 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:26.423845    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:26.441847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:26.441807    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7833-5866b3" podStartSLOduration=1.441794086 podStartE2EDuration="1.441794086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:26.440976685 +0000 UTC m=+1151.652169941" watchObservedRunningTime="2026-09-10 13:36:26.441794086 +0000 UTC m=+1151.652987397"
Sep 10 13:36:26.443273 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:26.443251    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:27.110402 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.110371    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7833-5866b3"]
Sep 10 13:36:27.119657 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.119599    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7833-5866b3"]
Sep 10 13:36:27.248883 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.248857    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7833-9a5f36"]
Sep 10 13:36:27.249180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.249168    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3e3a2c-4325-475a-808e-c6ec67fa1bce" containerName="test-curl-7833-5866b3"
Sep 10 13:36:27.249232 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.249182    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3e3a2c-4325-475a-808e-c6ec67fa1bce" containerName="test-curl-7833-5866b3"
Sep 10 13:36:27.249269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.249261    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb3e3a2c-4325-475a-808e-c6ec67fa1bce" containerName="test-curl-7833-5866b3"
Sep 10 13:36:27.252203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.252187    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-9a5f36"
Sep 10 13:36:27.257147 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.257124    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7833-9a5f36"]
Sep 10 13:36:27.350174 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.350145    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctlzq\" (UniqueName: \"kubernetes.io/projected/6b04e549-9886-49b8-87ab-f080f7efbf95-kube-api-access-ctlzq\") pod \"test-curl-7833-9a5f36\" (UID: \"6b04e549-9886-49b8-87ab-f080f7efbf95\") " pod="opendatahub/test-curl-7833-9a5f36"
Sep 10 13:36:27.426130 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:27.426067    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7833-5866b3" container="test-curl-7833-5866b3"
Sep 10 13:36:27.451356 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.451332    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctlzq\" (UniqueName: \"kubernetes.io/projected/6b04e549-9886-49b8-87ab-f080f7efbf95-kube-api-access-ctlzq\") pod \"test-curl-7833-9a5f36\" (UID: \"6b04e549-9886-49b8-87ab-f080f7efbf95\") " pod="opendatahub/test-curl-7833-9a5f36"
Sep 10 13:36:27.462400 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.462377    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctlzq\" (UniqueName: \"kubernetes.io/projected/6b04e549-9886-49b8-87ab-f080f7efbf95-kube-api-access-ctlzq\") pod \"test-curl-7833-9a5f36\" (UID: \"6b04e549-9886-49b8-87ab-f080f7efbf95\") " pod="opendatahub/test-curl-7833-9a5f36"
Sep 10 13:36:27.562577 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.562552    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-9a5f36"
Sep 10 13:36:27.835245 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:27.835222    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7833-9a5f36"]
Sep 10 13:36:27.837219 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:36:27.837190    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b04e549_9886_49b8_87ab_f080f7efbf95.slice/crio-d1ae2a8af824cd2126f8cd9c28c09f236c67cf1ee30a5abe1c82602d9e4638b2 WatchSource:0}: Error finding container d1ae2a8af824cd2126f8cd9c28c09f236c67cf1ee30a5abe1c82602d9e4638b2: Status 404 returned error can't find the container with id d1ae2a8af824cd2126f8cd9c28c09f236c67cf1ee30a5abe1c82602d9e4638b2
Sep 10 13:36:28.157754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.157681    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:28.432425 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.432400    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7833-9a5f36" event={"ID":"6b04e549-9886-49b8-87ab-f080f7efbf95","Type":"ContainerStarted","Data":"6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2"}
Sep 10 13:36:28.432720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.432439    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7833-9a5f36" event={"ID":"6b04e549-9886-49b8-87ab-f080f7efbf95","Type":"ContainerStarted","Data":"d1ae2a8af824cd2126f8cd9c28c09f236c67cf1ee30a5abe1c82602d9e4638b2"}
Sep 10 13:36:28.432720 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.432483    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7833-5866b3" podUID="bb3e3a2c-4325-475a-808e-c6ec67fa1bce" containerName="test-curl-7833-5866b3" containerID="cri-o://2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7" gracePeriod=2
Sep 10 13:36:28.450902 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.450859    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7833-9a5f36" podStartSLOduration=1.4508448010000001 podStartE2EDuration="1.450844801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:28.449457351 +0000 UTC m=+1153.660650608" watchObservedRunningTime="2026-09-10 13:36:28.450844801 +0000 UTC m=+1153.662038059"
Sep 10 13:36:28.612807 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.612778    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:28.616173 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:28.616144    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:28.659548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.659529    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvklm\" (UniqueName: \"kubernetes.io/projected/54c9b80c-5741-4a0f-926e-ead29e8addaf-kube-api-access-fvklm\") pod \"54c9b80c-5741-4a0f-926e-ead29e8addaf\" (UID: \"54c9b80c-5741-4a0f-926e-ead29e8addaf\") "
Sep 10 13:36:28.661548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.661521    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c9b80c-5741-4a0f-926e-ead29e8addaf-kube-api-access-fvklm" (OuterVolumeSpecName: "kube-api-access-fvklm") pod "54c9b80c-5741-4a0f-926e-ead29e8addaf" (UID: "54c9b80c-5741-4a0f-926e-ead29e8addaf"). InnerVolumeSpecName "kube-api-access-fvklm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:28.761180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:28.761128    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvklm\" (UniqueName: \"kubernetes.io/projected/54c9b80c-5741-4a0f-926e-ead29e8addaf-kube-api-access-fvklm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:36:29.103277 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.103251    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7833-9a5f36"]
Sep 10 13:36:29.107272 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.107248    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7833-9a5f36"]
Sep 10 13:36:29.248432 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.248403    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" path="/var/lib/kubelet/pods/54c9b80c-5741-4a0f-926e-ead29e8addaf/volumes"
Sep 10 13:36:29.438481 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.438397    2551 generic.go:356] "Generic (PLEG): container finished" podID="54c9b80c-5741-4a0f-926e-ead29e8addaf" containerID="8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717" exitCode=137
Sep 10 13:36:29.438933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.438491    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:29.438933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.438552    2551 scope.go:117] "RemoveContainer" containerID="8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717"
Sep 10 13:36:29.438933 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:29.438717    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7833-9a5f36" container="test-curl-7833-9a5f36"
Sep 10 13:36:29.441263 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:29.441220    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:29.443654 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:29.443627    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:29.446563 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.446546    2551 scope.go:117] "RemoveContainer" containerID="8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717"
Sep 10 13:36:29.446822 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:29.446804    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717\": container with ID starting with 8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717 not found: ID does not exist" containerID="8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717"
Sep 10 13:36:29.446890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.446831    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717"} err="failed to get container status \"8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717\": rpc error: code = NotFound desc = could not find container \"8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717\": container with ID starting with 8da0e3a2facf535875482aa121a5e06ce27abeca9f5f046c3c0984398cdf8717 not found: ID does not exist"
Sep 10 13:36:29.486870 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:29.486849    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:36:30.444003 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:30.443968    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7833-9a5f36" podUID="6b04e549-9886-49b8-87ab-f080f7efbf95" containerName="test-curl-7833-9a5f36" containerID="cri-o://6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2" gracePeriod=2
Sep 10 13:36:30.446651 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:30.446626    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:30.608600 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:30.608574    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-5866b3"
Sep 10 13:36:30.611178 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:30.611151    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7833-1b32e0\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="54c9b80c-5741-4a0f-926e-ead29e8addaf" pod="opendatahub/test-curl-7833-1b32e0"
Sep 10 13:36:30.679530 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:30.679502    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm57x\" (UniqueName: \"kubernetes.io/projected/bb3e3a2c-4325-475a-808e-c6ec67fa1bce-kube-api-access-cm57x\") pod \"bb3e3a2c-4325-475a-808e-c6ec67fa1bce\" (UID: \"bb3e3a2c-4325-475a-808e-c6ec67fa1bce\") "
Sep 10 13:36:30.681452 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:30.681424    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3e3a2c-4325-475a-808e-c6ec67fa1bce-kube-api-access-cm57x" (OuterVolumeSpecName: "kube-api-access-cm57x") pod "bb3e3a2c-4325-475a-808e-c6ec67fa1bce" (UID: "bb3e3a2c-4325-475a-808e-c6ec67fa1bce"). InnerVolumeSpecName "kube-api-access-cm57x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:30.780228 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:30.780200    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm57x\" (UniqueName: \"kubernetes.io/projected/bb3e3a2c-4325-475a-808e-c6ec67fa1bce-kube-api-access-cm57x\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:36:31.248445 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:31.248414    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb3e3a2c-4325-475a-808e-c6ec67fa1bce" path="/var/lib/kubelet/pods/bb3e3a2c-4325-475a-808e-c6ec67fa1bce/volumes"
Sep 10 13:36:31.449783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:31.449753    2551 generic.go:356] "Generic (PLEG): container finished" podID="bb3e3a2c-4325-475a-808e-c6ec67fa1bce" containerID="2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7" exitCode=137
Sep 10 13:36:31.450169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:31.449792    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-5866b3"
Sep 10 13:36:31.450169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:31.449852    2551 scope.go:117] "RemoveContainer" containerID="2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7"
Sep 10 13:36:31.457956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:31.457934    2551 scope.go:117] "RemoveContainer" containerID="2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7"
Sep 10 13:36:31.458207 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:31.458192    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7\": container with ID starting with 2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7 not found: ID does not exist" containerID="2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7"
Sep 10 13:36:31.458254 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:31.458215    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7"} err="failed to get container status \"2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7\": rpc error: code = NotFound desc = could not find container \"2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7\": container with ID starting with 2bef49499913775239baafad1966e0cd0708eea80a327e0aae3aded8d00072f7 not found: ID does not exist"
Sep 10 13:36:32.622422 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:32.622401    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-9a5f36"
Sep 10 13:36:32.694178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:32.694156    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctlzq\" (UniqueName: \"kubernetes.io/projected/6b04e549-9886-49b8-87ab-f080f7efbf95-kube-api-access-ctlzq\") pod \"6b04e549-9886-49b8-87ab-f080f7efbf95\" (UID: \"6b04e549-9886-49b8-87ab-f080f7efbf95\") "
Sep 10 13:36:32.695997 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:32.695966    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b04e549-9886-49b8-87ab-f080f7efbf95-kube-api-access-ctlzq" (OuterVolumeSpecName: "kube-api-access-ctlzq") pod "6b04e549-9886-49b8-87ab-f080f7efbf95" (UID: "6b04e549-9886-49b8-87ab-f080f7efbf95"). InnerVolumeSpecName "kube-api-access-ctlzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:36:32.795624 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:32.795601    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctlzq\" (UniqueName: \"kubernetes.io/projected/6b04e549-9886-49b8-87ab-f080f7efbf95-kube-api-access-ctlzq\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:36:33.251799 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:33.250212    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b04e549-9886-49b8-87ab-f080f7efbf95" path="/var/lib/kubelet/pods/6b04e549-9886-49b8-87ab-f080f7efbf95/volumes"
Sep 10 13:36:33.461302 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:33.461267    2551 generic.go:356] "Generic (PLEG): container finished" podID="6b04e549-9886-49b8-87ab-f080f7efbf95" containerID="6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2" exitCode=137
Sep 10 13:36:33.461450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:33.461307    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7833-9a5f36"
Sep 10 13:36:33.461450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:33.461369    2551 scope.go:117] "RemoveContainer" containerID="6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2"
Sep 10 13:36:33.469155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:33.469128    2551 scope.go:117] "RemoveContainer" containerID="6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2"
Sep 10 13:36:33.469440 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:36:33.469421    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2\": container with ID starting with 6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2 not found: ID does not exist" containerID="6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2"
Sep 10 13:36:33.469495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:33.469448    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2"} err="failed to get container status \"6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2\": rpc error: code = NotFound desc = could not find container \"6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2\": container with ID starting with 6c9053c4ac77371603da71e9f2225005bd40b2d98432dce810537d959417c9e2 not found: ID does not exist"
Sep 10 13:36:34.689232 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:34.689198    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:34.756825 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:34.756795    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:35.257495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:35.257465    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:37.974632 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:37.974559    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:38.080443 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:38.080416    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:39.322562 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.322528    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:39.421418 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.421384    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:39.744583 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.744495    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:39.846805 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.846768    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:39.947018 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.946984    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:39.973042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.973013    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"]
Sep 10 13:36:39.973382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.973370    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b04e549-9886-49b8-87ab-f080f7efbf95" containerName="test-curl-7833-9a5f36"
Sep 10 13:36:39.973431 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.973384    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b04e549-9886-49b8-87ab-f080f7efbf95" containerName="test-curl-7833-9a5f36"
Sep 10 13:36:39.973466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.973438    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b04e549-9886-49b8-87ab-f080f7efbf95" containerName="test-curl-7833-9a5f36"
Sep 10 13:36:39.977423 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.977407    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:39.980272 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.980256    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3ca6fbf6-dockercfg-g6cbv\""
Sep 10 13:36:39.980381 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.980332    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3ca6fbf6\""
Sep 10 13:36:39.985770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:39.985743    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"]
Sep 10 13:36:40.052493 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.052463    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1042aac-324a-4b75-8f43-73ce7011553e-plugins-config-volume\") pod \"payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") " pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:40.052622 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.052543    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fb7t\" (UniqueName: \"kubernetes.io/projected/e1042aac-324a-4b75-8f43-73ce7011553e-kube-api-access-5fb7t\") pod \"payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") " pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:40.153066 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.153034    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fb7t\" (UniqueName: \"kubernetes.io/projected/e1042aac-324a-4b75-8f43-73ce7011553e-kube-api-access-5fb7t\") pod \"payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") " pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:40.153236 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.153104    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1042aac-324a-4b75-8f43-73ce7011553e-plugins-config-volume\") pod \"payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") " pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:40.153750 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.153732    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1042aac-324a-4b75-8f43-73ce7011553e-plugins-config-volume\") pod \"payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") " pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:40.161977 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.161950    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fb7t\" (UniqueName: \"kubernetes.io/projected/e1042aac-324a-4b75-8f43-73ce7011553e-kube-api-access-5fb7t\") pod \"payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") " pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:40.288457 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.288423    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:40.563244 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:40.563168    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"]
Sep 10 13:36:40.565655 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:36:40.565625    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1042aac_324a_4b75_8f43_73ce7011553e.slice/crio-5d0ef0977684bf64380c1706a9ef72841c9a3884b56dd8dc3c2cad602c78d506 WatchSource:0}: Error finding container 5d0ef0977684bf64380c1706a9ef72841c9a3884b56dd8dc3c2cad602c78d506: Status 404 returned error can't find the container with id 5d0ef0977684bf64380c1706a9ef72841c9a3884b56dd8dc3c2cad602c78d506
Sep 10 13:36:41.499607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:41.499574    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb" event={"ID":"e1042aac-324a-4b75-8f43-73ce7011553e","Type":"ContainerStarted","Data":"d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578"}
Sep 10 13:36:41.499788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:41.499635    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:41.499788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:41.499660    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb" event={"ID":"e1042aac-324a-4b75-8f43-73ce7011553e","Type":"ContainerStarted","Data":"5d0ef0977684bf64380c1706a9ef72841c9a3884b56dd8dc3c2cad602c78d506"}
Sep 10 13:36:41.519385 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:41.519339    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb" podStartSLOduration=2.5193237330000002 podStartE2EDuration="2.519323733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:36:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:36:41.519135434 +0000 UTC m=+1166.730328692" watchObservedRunningTime="2026-09-10 13:36:41.519323733 +0000 UTC m=+1166.730516997"
Sep 10 13:36:44.795715 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:44.795676    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:36:47.896197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:47.896160    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:50.824445 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:50.824411    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:52.507126 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:52.507096    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:36:55.772108 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:55.772076    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:36:55.874009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:36:55.873981    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:00.333955 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:00.333922    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:03.025936 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:03.025906    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:04.483371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:04.483338    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:37:09.824517 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.824483    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"]
Sep 10 13:37:09.829308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.829282    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:09.833702 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.833677    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3ca6fbf6\"/\"default-dockercfg-mskkp\""
Sep 10 13:37:09.834958 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.834937    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3ca6fbf6\"/\"openshift-service-ca.crt\""
Sep 10 13:37:09.835078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.835038    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3ca6fbf6\"/\"e2e-lifecycle-model-3ca6fbf6-kserve-self-signed-certs\""
Sep 10 13:37:09.835142 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.835121    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3ca6fbf6\"/\"kube-root-ca.crt\""
Sep 10 13:37:09.840948 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.840927    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"]
Sep 10 13:37:09.860697 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:09.860671    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:10.003894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.003861    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-home\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.004009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.003921    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-model-cache\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.004009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.003973    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-dshm\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.004009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.004004    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-tmp-dir\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.004162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.004048    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbtcx\" (UniqueName: \"kubernetes.io/projected/447201e3-b062-4bb1-a322-4b5cd5f88365-kube-api-access-gbtcx\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.004162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.004076    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/447201e3-b062-4bb1-a322-4b5cd5f88365-tls-certs\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105282    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-model-cache\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105312    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-dshm\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105333    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-tmp-dir\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105373    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbtcx\" (UniqueName: \"kubernetes.io/projected/447201e3-b062-4bb1-a322-4b5cd5f88365-kube-api-access-gbtcx\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105402    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/447201e3-b062-4bb1-a322-4b5cd5f88365-tls-certs\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105447    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-home\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105713 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105644    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-model-cache\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105788 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105769    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-tmp-dir\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.105833 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.105805    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-home\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.107490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.107468    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-dshm\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.107879 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.107861    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/447201e3-b062-4bb1-a322-4b5cd5f88365-tls-certs\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.115079 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.115060    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbtcx\" (UniqueName: \"kubernetes.io/projected/447201e3-b062-4bb1-a322-4b5cd5f88365-kube-api-access-gbtcx\") pod \"e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") " pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.138395 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.138369    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:10.415074 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.415051    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"]
Sep 10 13:37:10.416834 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:37:10.416811    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod447201e3_b062_4bb1_a322_4b5cd5f88365.slice/crio-66e76386855a3e76ffdce43cd81eea36b977a477430890f7f8526fdc7c5b9479 WatchSource:0}: Error finding container 66e76386855a3e76ffdce43cd81eea36b977a477430890f7f8526fdc7c5b9479: Status 404 returned error can't find the container with id 66e76386855a3e76ffdce43cd81eea36b977a477430890f7f8526fdc7c5b9479
Sep 10 13:37:10.638220 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.638189    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm" event={"ID":"447201e3-b062-4bb1-a322-4b5cd5f88365","Type":"ContainerStarted","Data":"e0d404e1616becc87c73f1b90fe2c36df6a9c8987b0fb8688db3ac564c363fcc"}
Sep 10 13:37:10.638406 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.638236    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm" event={"ID":"447201e3-b062-4bb1-a322-4b5cd5f88365","Type":"ContainerStarted","Data":"66e76386855a3e76ffdce43cd81eea36b977a477430890f7f8526fdc7c5b9479"}
Sep 10 13:37:10.659491 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:10.659446    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm" podStartSLOduration=1.659429554 podStartE2EDuration="1.659429554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:37:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:10.658942053 +0000 UTC m=+1195.870135311" watchObservedRunningTime="2026-09-10 13:37:10.659429554 +0000 UTC m=+1195.870622813"
Sep 10 13:37:15.220466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:15.220435    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:37:15.222393 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:15.222369    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:37:19.434452 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:19.434420    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:37:20.138929 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:20.138900    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:20.139111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:20.138941    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:20.151655 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:20.151626    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:20.698941 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:20.698910    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:37:31.717579 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:31.717544    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:33.085759 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:33.085719    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:37.444751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:37.444715    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:37:42.865106 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:42.865074    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"]
Sep 10 13:37:42.868939 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:42.868921    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"
Sep 10 13:37:42.875634 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:42.875592    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:37:42.875890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:42.875872    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:37:42.884833 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:42.884810    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"]
Sep 10 13:37:42.965794 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:42.965758    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e082b8f-26c9-470f-bb83-91be7b853e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t\" (UID: \"1e082b8f-26c9-470f-bb83-91be7b853e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"
Sep 10 13:37:43.066996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:43.066964    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e082b8f-26c9-470f-bb83-91be7b853e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t\" (UID: \"1e082b8f-26c9-470f-bb83-91be7b853e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"
Sep 10 13:37:43.067548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:43.067529    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e082b8f-26c9-470f-bb83-91be7b853e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t\" (UID: \"1e082b8f-26c9-470f-bb83-91be7b853e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"
Sep 10 13:37:43.178801 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:43.178742    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"
Sep 10 13:37:43.476069 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:43.476039    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"]
Sep 10 13:37:43.477448 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:37:43.477421    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e082b8f_26c9_470f_bb83_91be7b853e31.slice/crio-517a62980253fe9d96ea5152776c75755d0d20bba88e3f3c519b45e51ce13a9d WatchSource:0}: Error finding container 517a62980253fe9d96ea5152776c75755d0d20bba88e3f3c519b45e51ce13a9d: Status 404 returned error can't find the container with id 517a62980253fe9d96ea5152776c75755d0d20bba88e3f3c519b45e51ce13a9d
Sep 10 13:37:43.799454 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:43.799425    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t" event={"ID":"1e082b8f-26c9-470f-bb83-91be7b853e31","Type":"ContainerStarted","Data":"b273cfb9e7aaafdd6fffa9fd915a7bb05d774fa58bd5bfa5c517924bb768549f"}
Sep 10 13:37:43.799566 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:43.799482    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t" event={"ID":"1e082b8f-26c9-470f-bb83-91be7b853e31","Type":"ContainerStarted","Data":"517a62980253fe9d96ea5152776c75755d0d20bba88e3f3c519b45e51ce13a9d"}
Sep 10 13:37:43.820016 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:43.819971    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t" podStartSLOduration=1.8199562120000001 podStartE2EDuration="1.819956212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:37:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:37:43.819053193 +0000 UTC m=+1229.030246442" watchObservedRunningTime="2026-09-10 13:37:43.819956212 +0000 UTC m=+1229.031149469"
Sep 10 13:37:44.805284 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:44.805248    2551 generic.go:356] "Generic (PLEG): container finished" podID="1e082b8f-26c9-470f-bb83-91be7b853e31" containerID="b273cfb9e7aaafdd6fffa9fd915a7bb05d774fa58bd5bfa5c517924bb768549f" exitCode=0
Sep 10 13:37:44.805686 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:44.805337    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t" event={"ID":"1e082b8f-26c9-470f-bb83-91be7b853e31","Type":"ContainerDied","Data":"b273cfb9e7aaafdd6fffa9fd915a7bb05d774fa58bd5bfa5c517924bb768549f"}
Sep 10 13:37:45.939355 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:45.939330    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"
Sep 10 13:37:45.991712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:45.991654    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e082b8f-26c9-470f-bb83-91be7b853e31-maas-api-service-ca\") pod \"1e082b8f-26c9-470f-bb83-91be7b853e31\" (UID: \"1e082b8f-26c9-470f-bb83-91be7b853e31\") "
Sep 10 13:37:45.991975 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:45.991953    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e082b8f-26c9-470f-bb83-91be7b853e31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e082b8f-26c9-470f-bb83-91be7b853e31" (UID: "1e082b8f-26c9-470f-bb83-91be7b853e31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:37:46.093044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:46.092986    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e082b8f-26c9-470f-bb83-91be7b853e31-maas-api-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:37:46.816060 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:46.816030    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"
Sep 10 13:37:46.816213 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:46.816026    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t" event={"ID":"1e082b8f-26c9-470f-bb83-91be7b853e31","Type":"ContainerDied","Data":"517a62980253fe9d96ea5152776c75755d0d20bba88e3f3c519b45e51ce13a9d"}
Sep 10 13:37:46.816213 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:46.816163    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="517a62980253fe9d96ea5152776c75755d0d20bba88e3f3c519b45e51ce13a9d"
Sep 10 13:37:54.619903 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:54.619869    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:37:58.720578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:58.720547    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:58.842897 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:58.842865    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"]
Sep 10 13:37:58.875755 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:58.875715    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb" podUID="e1042aac-324a-4b75-8f43-73ce7011553e" containerName="payload-processing" containerID="cri-o://d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578" gracePeriod=30
Sep 10 13:37:58.896756 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:58.896725    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:59.033248 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.033218    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:59.121682 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.121635    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:37:59.197854 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.197830    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fb7t\" (UniqueName: \"kubernetes.io/projected/e1042aac-324a-4b75-8f43-73ce7011553e-kube-api-access-5fb7t\") pod \"e1042aac-324a-4b75-8f43-73ce7011553e\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") "
Sep 10 13:37:59.198011 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.197878    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1042aac-324a-4b75-8f43-73ce7011553e-plugins-config-volume\") pod \"e1042aac-324a-4b75-8f43-73ce7011553e\" (UID: \"e1042aac-324a-4b75-8f43-73ce7011553e\") "
Sep 10 13:37:59.198285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.198260    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1042aac-324a-4b75-8f43-73ce7011553e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e1042aac-324a-4b75-8f43-73ce7011553e" (UID: "e1042aac-324a-4b75-8f43-73ce7011553e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:37:59.199896 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.199870    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1042aac-324a-4b75-8f43-73ce7011553e-kube-api-access-5fb7t" (OuterVolumeSpecName: "kube-api-access-5fb7t") pod "e1042aac-324a-4b75-8f43-73ce7011553e" (UID: "e1042aac-324a-4b75-8f43-73ce7011553e"). InnerVolumeSpecName "kube-api-access-5fb7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:37:59.250531 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.250446    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:59.299497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.299468    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fb7t\" (UniqueName: \"kubernetes.io/projected/e1042aac-324a-4b75-8f43-73ce7011553e-kube-api-access-5fb7t\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:37:59.299497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.299498    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1042aac-324a-4b75-8f43-73ce7011553e-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:37:59.303171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.303142    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"]
Sep 10 13:37:59.309676 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.309640    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3ca6fbf6-q9t2t"]
Sep 10 13:37:59.344410 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.344384    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:37:59.881325 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.881286    2551 generic.go:356] "Generic (PLEG): container finished" podID="e1042aac-324a-4b75-8f43-73ce7011553e" containerID="d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578" exitCode=0
Sep 10 13:37:59.881744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.881340    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"
Sep 10 13:37:59.881744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.881362    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb" event={"ID":"e1042aac-324a-4b75-8f43-73ce7011553e","Type":"ContainerDied","Data":"d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578"}
Sep 10 13:37:59.881744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.881433    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb" event={"ID":"e1042aac-324a-4b75-8f43-73ce7011553e","Type":"ContainerDied","Data":"5d0ef0977684bf64380c1706a9ef72841c9a3884b56dd8dc3c2cad602c78d506"}
Sep 10 13:37:59.881744 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.881455    2551 scope.go:117] "RemoveContainer" containerID="d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578"
Sep 10 13:37:59.890039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.890019    2551 scope.go:117] "RemoveContainer" containerID="d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578"
Sep 10 13:37:59.890314 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:37:59.890295    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578\": container with ID starting with d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578 not found: ID does not exist" containerID="d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578"
Sep 10 13:37:59.890368 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.890323    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578"} err="failed to get container status \"d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578\": rpc error: code = NotFound desc = could not find container \"d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578\": container with ID starting with d801f40120c88e9493a640c2ffe306ad5abd837fbbda81d9fe1fdd2d1d5df578 not found: ID does not exist"
Sep 10 13:37:59.913560 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.913536    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"]
Sep 10 13:37:59.918894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:37:59.918873    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3ca6fbf6-5487cf5589-x84cb"]
Sep 10 13:38:01.248329 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:01.248292    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e082b8f-26c9-470f-bb83-91be7b853e31" path="/var/lib/kubelet/pods/1e082b8f-26c9-470f-bb83-91be7b853e31/volumes"
Sep 10 13:38:01.248799 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:01.248684    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1042aac-324a-4b75-8f43-73ce7011553e" path="/var/lib/kubelet/pods/e1042aac-324a-4b75-8f43-73ce7011553e/volumes"
Sep 10 13:38:02.779732 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:02.779697    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:38:07.760943 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:07.760911    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"]
Sep 10 13:38:07.868012 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:07.867921    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:38:12.218896 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:12.218864    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:38:15.374804 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:15.374772    2551 scope.go:117] "RemoveContainer" containerID="8e5badfbf3cb08be2cc4766c10c0482f1cccff10e448769bc8a0354af5167bd9"
Sep 10 13:38:21.720039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.719999    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7836-98d05a"]
Sep 10 13:38:21.720521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.720369    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e082b8f-26c9-470f-bb83-91be7b853e31" containerName="revoke-keys"
Sep 10 13:38:21.720521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.720381    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e082b8f-26c9-470f-bb83-91be7b853e31" containerName="revoke-keys"
Sep 10 13:38:21.720521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.720401    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1042aac-324a-4b75-8f43-73ce7011553e" containerName="payload-processing"
Sep 10 13:38:21.720521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.720407    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1042aac-324a-4b75-8f43-73ce7011553e" containerName="payload-processing"
Sep 10 13:38:21.720521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.720469    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e082b8f-26c9-470f-bb83-91be7b853e31" containerName="revoke-keys"
Sep 10 13:38:21.720521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.720477    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1042aac-324a-4b75-8f43-73ce7011553e" containerName="payload-processing"
Sep 10 13:38:21.724731 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.724713    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:21.727581 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.727557    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tqk99\""
Sep 10 13:38:21.730303 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.730279    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7836-98d05a"]
Sep 10 13:38:21.801468 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.801430    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k24rq\" (UniqueName: \"kubernetes.io/projected/480b82cc-c75d-40b2-b18b-8aa1ef2abd26-kube-api-access-k24rq\") pod \"test-curl-7836-98d05a\" (UID: \"480b82cc-c75d-40b2-b18b-8aa1ef2abd26\") " pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:21.902494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.902458    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k24rq\" (UniqueName: \"kubernetes.io/projected/480b82cc-c75d-40b2-b18b-8aa1ef2abd26-kube-api-access-k24rq\") pod \"test-curl-7836-98d05a\" (UID: \"480b82cc-c75d-40b2-b18b-8aa1ef2abd26\") " pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:21.911893 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:21.911862    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k24rq\" (UniqueName: \"kubernetes.io/projected/480b82cc-c75d-40b2-b18b-8aa1ef2abd26-kube-api-access-k24rq\") pod \"test-curl-7836-98d05a\" (UID: \"480b82cc-c75d-40b2-b18b-8aa1ef2abd26\") " pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:22.036082 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:22.036058    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:22.314408 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:22.314385    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7836-98d05a"]
Sep 10 13:38:22.316843 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:38:22.316817    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod480b82cc_c75d_40b2_b18b_8aa1ef2abd26.slice/crio-4f1d5774d4603bbf4720577509836b950b9b888ac061b623a1f41558cb707f4d WatchSource:0}: Error finding container 4f1d5774d4603bbf4720577509836b950b9b888ac061b623a1f41558cb707f4d: Status 404 returned error can't find the container with id 4f1d5774d4603bbf4720577509836b950b9b888ac061b623a1f41558cb707f4d
Sep 10 13:38:22.780828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:22.780788    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm" podUID="447201e3-b062-4bb1-a322-4b5cd5f88365" containerName="main" containerID="cri-o://e0d404e1616becc87c73f1b90fe2c36df6a9c8987b0fb8688db3ac564c363fcc" gracePeriod=45
Sep 10 13:38:23.003901 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.003860    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7836-98d05a" event={"ID":"480b82cc-c75d-40b2-b18b-8aa1ef2abd26","Type":"ContainerStarted","Data":"8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279"}
Sep 10 13:38:23.003901 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.003909    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7836-98d05a" event={"ID":"480b82cc-c75d-40b2-b18b-8aa1ef2abd26","Type":"ContainerStarted","Data":"4f1d5774d4603bbf4720577509836b950b9b888ac061b623a1f41558cb707f4d"}
Sep 10 13:38:23.005635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.005605    2551 generic.go:356] "Generic (PLEG): container finished" podID="447201e3-b062-4bb1-a322-4b5cd5f88365" containerID="e0d404e1616becc87c73f1b90fe2c36df6a9c8987b0fb8688db3ac564c363fcc" exitCode=0
Sep 10 13:38:23.005773 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.005659    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm" event={"ID":"447201e3-b062-4bb1-a322-4b5cd5f88365","Type":"ContainerDied","Data":"e0d404e1616becc87c73f1b90fe2c36df6a9c8987b0fb8688db3ac564c363fcc"}
Sep 10 13:38:23.023828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.023785    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7836-98d05a" podStartSLOduration=2.023771674 podStartE2EDuration="2.023771674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:38:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:38:23.022434084 +0000 UTC m=+1268.233627355" watchObservedRunningTime="2026-09-10 13:38:23.023771674 +0000 UTC m=+1268.234964930"
Sep 10 13:38:23.041717 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.041641    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:38:23.113296 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113263    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-home\") pod \"447201e3-b062-4bb1-a322-4b5cd5f88365\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") "
Sep 10 13:38:23.113478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113306    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-dshm\") pod \"447201e3-b062-4bb1-a322-4b5cd5f88365\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") "
Sep 10 13:38:23.113478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113358    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/447201e3-b062-4bb1-a322-4b5cd5f88365-tls-certs\") pod \"447201e3-b062-4bb1-a322-4b5cd5f88365\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") "
Sep 10 13:38:23.113478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113386    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-model-cache\") pod \"447201e3-b062-4bb1-a322-4b5cd5f88365\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") "
Sep 10 13:38:23.113478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113429    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbtcx\" (UniqueName: \"kubernetes.io/projected/447201e3-b062-4bb1-a322-4b5cd5f88365-kube-api-access-gbtcx\") pod \"447201e3-b062-4bb1-a322-4b5cd5f88365\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") "
Sep 10 13:38:23.113722 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113491    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-tmp-dir\") pod \"447201e3-b062-4bb1-a322-4b5cd5f88365\" (UID: \"447201e3-b062-4bb1-a322-4b5cd5f88365\") "
Sep 10 13:38:23.113722 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113590    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-home" (OuterVolumeSpecName: "home") pod "447201e3-b062-4bb1-a322-4b5cd5f88365" (UID: "447201e3-b062-4bb1-a322-4b5cd5f88365"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:23.113824 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113742    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-model-cache" (OuterVolumeSpecName: "model-cache") pod "447201e3-b062-4bb1-a322-4b5cd5f88365" (UID: "447201e3-b062-4bb1-a322-4b5cd5f88365"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:23.113824 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113804    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:23.113824 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.113809    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "447201e3-b062-4bb1-a322-4b5cd5f88365" (UID: "447201e3-b062-4bb1-a322-4b5cd5f88365"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:23.115747 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.115719    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-dshm" (OuterVolumeSpecName: "dshm") pod "447201e3-b062-4bb1-a322-4b5cd5f88365" (UID: "447201e3-b062-4bb1-a322-4b5cd5f88365"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:23.115881 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.115847    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/447201e3-b062-4bb1-a322-4b5cd5f88365-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "447201e3-b062-4bb1-a322-4b5cd5f88365" (UID: "447201e3-b062-4bb1-a322-4b5cd5f88365"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:38:23.116063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.116038    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/447201e3-b062-4bb1-a322-4b5cd5f88365-kube-api-access-gbtcx" (OuterVolumeSpecName: "kube-api-access-gbtcx") pod "447201e3-b062-4bb1-a322-4b5cd5f88365" (UID: "447201e3-b062-4bb1-a322-4b5cd5f88365"). InnerVolumeSpecName "kube-api-access-gbtcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:38:23.214727 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.214699    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbtcx\" (UniqueName: \"kubernetes.io/projected/447201e3-b062-4bb1-a322-4b5cd5f88365-kube-api-access-gbtcx\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:23.214727 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.214723    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:23.214727 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.214733    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:23.214934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.214741    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/447201e3-b062-4bb1-a322-4b5cd5f88365-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:23.214934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.214749    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/447201e3-b062-4bb1-a322-4b5cd5f88365-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:23.663004 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.662970    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7836-98d05a"]
Sep 10 13:38:23.666955 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:23.666926    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7836-98d05a"]
Sep 10 13:38:24.012171 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:24.012135    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"
Sep 10 13:38:24.012642 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:24.012176    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm" event={"ID":"447201e3-b062-4bb1-a322-4b5cd5f88365","Type":"ContainerDied","Data":"66e76386855a3e76ffdce43cd81eea36b977a477430890f7f8526fdc7c5b9479"}
Sep 10 13:38:24.012642 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:24.012230    2551 scope.go:117] "RemoveContainer" containerID="e0d404e1616becc87c73f1b90fe2c36df6a9c8987b0fb8688db3ac564c363fcc"
Sep 10 13:38:24.012642 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:38:24.012455    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7836-98d05a" container="test-curl-7836-98d05a"
Sep 10 13:38:24.031385 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:24.031361    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"]
Sep 10 13:38:24.037542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:24.037518    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3ca6fbf6/e2e-lifecycle-model-3ca6fbf6-kserve-7bddd78c5-vkldm"]
Sep 10 13:38:25.024618 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:25.024576    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7836-98d05a" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" containerName="test-curl-7836-98d05a" containerID="cri-o://8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279" gracePeriod=2
Sep 10 13:38:25.027271 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:38:25.027240    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7836-98d05a\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:25.109703 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:25.109658    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"]
Sep 10 13:38:25.112593 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:38:25.112551    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7836-98d05a\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:25.133956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:25.133925    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:38:25.247753 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:38:25.247708    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7836-98d05a\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:25.249192 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:25.249171    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="447201e3-b062-4bb1-a322-4b5cd5f88365" path="/var/lib/kubelet/pods/447201e3-b062-4bb1-a322-4b5cd5f88365/volumes"
Sep 10 13:38:26.093807 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:26.093775    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"]
Sep 10 13:38:26.115347 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:26.115314    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:38:27.186910 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:27.186881    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:27.252017 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:27.251993    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k24rq\" (UniqueName: \"kubernetes.io/projected/480b82cc-c75d-40b2-b18b-8aa1ef2abd26-kube-api-access-k24rq\") pod \"480b82cc-c75d-40b2-b18b-8aa1ef2abd26\" (UID: \"480b82cc-c75d-40b2-b18b-8aa1ef2abd26\") "
Sep 10 13:38:27.254039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:27.254012    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/480b82cc-c75d-40b2-b18b-8aa1ef2abd26-kube-api-access-k24rq" (OuterVolumeSpecName: "kube-api-access-k24rq") pod "480b82cc-c75d-40b2-b18b-8aa1ef2abd26" (UID: "480b82cc-c75d-40b2-b18b-8aa1ef2abd26"). InnerVolumeSpecName "kube-api-access-k24rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:38:27.353190 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:27.353100    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k24rq\" (UniqueName: \"kubernetes.io/projected/480b82cc-c75d-40b2-b18b-8aa1ef2abd26-kube-api-access-k24rq\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:28.040425 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:28.040391    2551 generic.go:356] "Generic (PLEG): container finished" podID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" containerID="8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279" exitCode=137
Sep 10 13:38:28.040425 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:28.040430    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-98d05a"
Sep 10 13:38:28.040700 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:28.040469    2551 scope.go:117] "RemoveContainer" containerID="8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279"
Sep 10 13:38:28.049220 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:28.049202    2551 scope.go:117] "RemoveContainer" containerID="8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279"
Sep 10 13:38:28.049497 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:38:28.049476    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279\": container with ID starting with 8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279 not found: ID does not exist" containerID="8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279"
Sep 10 13:38:28.049594 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:28.049505    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279"} err="failed to get container status \"8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279\": rpc error: code = NotFound desc = could not find container \"8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279\": container with ID starting with 8a0b668f18237b2ab3006b2178eea9e7c83ea8e5eee3aaaa573fefc6820b2279 not found: ID does not exist"
Sep 10 13:38:29.248847 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:29.248816    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" path="/var/lib/kubelet/pods/480b82cc-c75d-40b2-b18b-8aa1ef2abd26/volumes"
Sep 10 13:38:29.501898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:29.501803    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:38:34.118295 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.118263    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"]
Sep 10 13:38:34.118695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.118603    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" containerName="test-curl-7836-98d05a"
Sep 10 13:38:34.118695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.118614    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" containerName="test-curl-7836-98d05a"
Sep 10 13:38:34.118695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.118635    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="447201e3-b062-4bb1-a322-4b5cd5f88365" containerName="main"
Sep 10 13:38:34.118695 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.118641    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="447201e3-b062-4bb1-a322-4b5cd5f88365" containerName="main"
Sep 10 13:38:34.118865 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.118720    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="447201e3-b062-4bb1-a322-4b5cd5f88365" containerName="main"
Sep 10 13:38:34.118865 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.118731    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="480b82cc-c75d-40b2-b18b-8aa1ef2abd26" containerName="test-curl-7836-98d05a"
Sep 10 13:38:34.123070 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.123050    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.126783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.126764    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7657ef\"/\"sub-test-model-7657ef-kserve-self-signed-certs\""
Sep 10 13:38:34.172612 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.172589    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"]
Sep 10 13:38:34.194649 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.194620    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:38:34.213890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.213863    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-dshm\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.214006 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.213905    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tls-certs\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.214006 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.213927    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-model-cache\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.214135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.214006    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-home\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.214135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.214041    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfhx2\" (UniqueName: \"kubernetes.io/projected/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-kube-api-access-cfhx2\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.214135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.214117    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tmp-dir\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315184 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315150    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-model-cache\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315304 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315217    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-home\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315304 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315248    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfhx2\" (UniqueName: \"kubernetes.io/projected/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-kube-api-access-cfhx2\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315304 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315281    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tmp-dir\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315471 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315407    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-dshm\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315527 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315472    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tls-certs\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315585 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315554    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-model-cache\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315715 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315693    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tmp-dir\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.315968 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.315949    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-home\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.317618 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.317599    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-dshm\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.317920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.317903    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tls-certs\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.326298 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.326277    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfhx2\" (UniqueName: \"kubernetes.io/projected/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-kube-api-access-cfhx2\") pod \"sub-test-model-7657ef-kserve-66f45d5579-p26gh\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.431954 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.431901    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:34.706515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:34.706489    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"]
Sep 10 13:38:34.708923 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:38:34.708893    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e0cd0fa_33ec_46a6_9cb7_e7fd2c23671e.slice/crio-0258c358ac7f5f10612b6e45b2ca405909aea866b81fac7684fcc34babf8cef7 WatchSource:0}: Error finding container 0258c358ac7f5f10612b6e45b2ca405909aea866b81fac7684fcc34babf8cef7: Status 404 returned error can't find the container with id 0258c358ac7f5f10612b6e45b2ca405909aea866b81fac7684fcc34babf8cef7
Sep 10 13:38:35.076281 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:35.076248    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh" event={"ID":"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e","Type":"ContainerStarted","Data":"c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6"}
Sep 10 13:38:35.076440 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:35.076288    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh" event={"ID":"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e","Type":"ContainerStarted","Data":"0258c358ac7f5f10612b6e45b2ca405909aea866b81fac7684fcc34babf8cef7"}
Sep 10 13:38:35.098861 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:35.098817    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh" podStartSLOduration=1.098802778 podStartE2EDuration="1.098802778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:38:35.09708193 +0000 UTC m=+1280.308275202" watchObservedRunningTime="2026-09-10 13:38:35.098802778 +0000 UTC m=+1280.309996035"
Sep 10 13:38:40.128967 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.128920    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq" podUID="599996d1-3cdb-431d-b6ad-dd1ba1d5e999" containerName="main" containerID="cri-o://f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46" gracePeriod=45
Sep 10 13:38:40.375543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.375520    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:38:40.471787 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.471717    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-model-cache\") pod \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") "
Sep 10 13:38:40.471916 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.471832    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-dshm\") pod \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") "
Sep 10 13:38:40.471916 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.471868    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjbgn\" (UniqueName: \"kubernetes.io/projected/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-kube-api-access-sjbgn\") pod \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") "
Sep 10 13:38:40.471916 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.471903    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tmp-dir\") pod \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") "
Sep 10 13:38:40.472088 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.471930    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-home\") pod \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") "
Sep 10 13:38:40.472088 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.471941    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-model-cache" (OuterVolumeSpecName: "model-cache") pod "599996d1-3cdb-431d-b6ad-dd1ba1d5e999" (UID: "599996d1-3cdb-431d-b6ad-dd1ba1d5e999"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:40.472088 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.472002    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tls-certs\") pod \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\" (UID: \"599996d1-3cdb-431d-b6ad-dd1ba1d5e999\") "
Sep 10 13:38:40.472225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.472109    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "599996d1-3cdb-431d-b6ad-dd1ba1d5e999" (UID: "599996d1-3cdb-431d-b6ad-dd1ba1d5e999"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:40.472282 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.472241    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-home" (OuterVolumeSpecName: "home") pod "599996d1-3cdb-431d-b6ad-dd1ba1d5e999" (UID: "599996d1-3cdb-431d-b6ad-dd1ba1d5e999"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:40.472330 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.472279    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:40.472330 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.472295    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:40.473972 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.473950    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-dshm" (OuterVolumeSpecName: "dshm") pod "599996d1-3cdb-431d-b6ad-dd1ba1d5e999" (UID: "599996d1-3cdb-431d-b6ad-dd1ba1d5e999"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:40.474068 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.473988    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-kube-api-access-sjbgn" (OuterVolumeSpecName: "kube-api-access-sjbgn") pod "599996d1-3cdb-431d-b6ad-dd1ba1d5e999" (UID: "599996d1-3cdb-431d-b6ad-dd1ba1d5e999"). InnerVolumeSpecName "kube-api-access-sjbgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:38:40.474133 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.474063    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "599996d1-3cdb-431d-b6ad-dd1ba1d5e999" (UID: "599996d1-3cdb-431d-b6ad-dd1ba1d5e999"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:38:40.573104 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.573073    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:40.573104 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.573100    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjbgn\" (UniqueName: \"kubernetes.io/projected/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-kube-api-access-sjbgn\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:40.573259 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.573114    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:40.573259 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:40.573126    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/599996d1-3cdb-431d-b6ad-dd1ba1d5e999-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:41.107301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.107266    2551 generic.go:356] "Generic (PLEG): container finished" podID="599996d1-3cdb-431d-b6ad-dd1ba1d5e999" containerID="f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46" exitCode=0
Sep 10 13:38:41.107489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.107311    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"
Sep 10 13:38:41.107489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.107344    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq" event={"ID":"599996d1-3cdb-431d-b6ad-dd1ba1d5e999","Type":"ContainerDied","Data":"f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46"}
Sep 10 13:38:41.107489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.107385    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq" event={"ID":"599996d1-3cdb-431d-b6ad-dd1ba1d5e999","Type":"ContainerDied","Data":"7bb43a9582866c4c84f10d3ea1f499d2e522c5b370d9942ff246571f2dfc6943"}
Sep 10 13:38:41.107489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.107404    2551 scope.go:117] "RemoveContainer" containerID="f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46"
Sep 10 13:38:41.113001 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.112958    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs" podUID="9217f8be-4e16-4dea-96d9-e46e7c21fe23" containerName="main" containerID="cri-o://fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2" gracePeriod=45
Sep 10 13:38:41.116783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.116685    2551 scope.go:117] "RemoveContainer" containerID="f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46"
Sep 10 13:38:41.116972 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:38:41.116945    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46\": container with ID starting with f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46 not found: ID does not exist" containerID="f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46"
Sep 10 13:38:41.117054 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.116978    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46"} err="failed to get container status \"f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46\": rpc error: code = NotFound desc = could not find container \"f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46\": container with ID starting with f0643fa2b618e42168f06bb282c899f25085af38de1cdce3f817b2d49cec5d46 not found: ID does not exist"
Sep 10 13:38:41.131050 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.131029    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"]
Sep 10 13:38:41.135650 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.135626    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/auth-test-model-7657ef-kserve-76596fd9bc-frdvq"]
Sep 10 13:38:41.249269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.249242    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="599996d1-3cdb-431d-b6ad-dd1ba1d5e999" path="/var/lib/kubelet/pods/599996d1-3cdb-431d-b6ad-dd1ba1d5e999/volumes"
Sep 10 13:38:41.367635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.367582    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:38:41.481754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.481727    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw25b\" (UniqueName: \"kubernetes.io/projected/9217f8be-4e16-4dea-96d9-e46e7c21fe23-kube-api-access-kw25b\") pod \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") "
Sep 10 13:38:41.481905 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.481760    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tmp-dir\") pod \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") "
Sep 10 13:38:41.481905 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.481805    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tls-certs\") pod \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") "
Sep 10 13:38:41.481905 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.481836    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-dshm\") pod \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") "
Sep 10 13:38:41.481905 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.481864    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-model-cache\") pod \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") "
Sep 10 13:38:41.481905 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.481903    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-home\") pod \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\" (UID: \"9217f8be-4e16-4dea-96d9-e46e7c21fe23\") "
Sep 10 13:38:41.482162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.482035    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9217f8be-4e16-4dea-96d9-e46e7c21fe23" (UID: "9217f8be-4e16-4dea-96d9-e46e7c21fe23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:41.482162 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.482119    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-model-cache" (OuterVolumeSpecName: "model-cache") pod "9217f8be-4e16-4dea-96d9-e46e7c21fe23" (UID: "9217f8be-4e16-4dea-96d9-e46e7c21fe23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:41.482275 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.482199    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-home" (OuterVolumeSpecName: "home") pod "9217f8be-4e16-4dea-96d9-e46e7c21fe23" (UID: "9217f8be-4e16-4dea-96d9-e46e7c21fe23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:41.484027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.484004    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-dshm" (OuterVolumeSpecName: "dshm") pod "9217f8be-4e16-4dea-96d9-e46e7c21fe23" (UID: "9217f8be-4e16-4dea-96d9-e46e7c21fe23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:38:41.484119 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.484050    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9217f8be-4e16-4dea-96d9-e46e7c21fe23" (UID: "9217f8be-4e16-4dea-96d9-e46e7c21fe23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:38:41.484119 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.484050    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9217f8be-4e16-4dea-96d9-e46e7c21fe23-kube-api-access-kw25b" (OuterVolumeSpecName: "kube-api-access-kw25b") pod "9217f8be-4e16-4dea-96d9-e46e7c21fe23" (UID: "9217f8be-4e16-4dea-96d9-e46e7c21fe23"). InnerVolumeSpecName "kube-api-access-kw25b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:38:41.583076 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.583049    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw25b\" (UniqueName: \"kubernetes.io/projected/9217f8be-4e16-4dea-96d9-e46e7c21fe23-kube-api-access-kw25b\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:41.583076 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.583073    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:41.583076 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.583083    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9217f8be-4e16-4dea-96d9-e46e7c21fe23-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:41.583272 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.583090    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:41.583272 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.583098    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:41.583272 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:41.583106    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9217f8be-4e16-4dea-96d9-e46e7c21fe23-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:38:42.114738 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.114703    2551 generic.go:356] "Generic (PLEG): container finished" podID="9217f8be-4e16-4dea-96d9-e46e7c21fe23" containerID="fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2" exitCode=0
Sep 10 13:38:42.114895 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.114741    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"
Sep 10 13:38:42.114895 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.114767    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs" event={"ID":"9217f8be-4e16-4dea-96d9-e46e7c21fe23","Type":"ContainerDied","Data":"fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2"}
Sep 10 13:38:42.114895 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.114798    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs" event={"ID":"9217f8be-4e16-4dea-96d9-e46e7c21fe23","Type":"ContainerDied","Data":"513439a5986cafc563a4f1f3d142966fd9d3a19e845086bc37611cf0584e69e5"}
Sep 10 13:38:42.114895 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.114814    2551 scope.go:117] "RemoveContainer" containerID="fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2"
Sep 10 13:38:42.123564 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.123548    2551 scope.go:117] "RemoveContainer" containerID="fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2"
Sep 10 13:38:42.123828 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:38:42.123809    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2\": container with ID starting with fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2 not found: ID does not exist" containerID="fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2"
Sep 10 13:38:42.123892 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.123836    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2"} err="failed to get container status \"fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2\": rpc error: code = NotFound desc = could not find container \"fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2\": container with ID starting with fd7577c5d37a3f8b62eb030a73c54040b7d844c5eb0618efbc2b0858e6d113c2 not found: ID does not exist"
Sep 10 13:38:42.153865 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.153842    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"]
Sep 10 13:38:42.157712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:42.157692    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/auth-test-model-7739cb-kserve-6698796879-xpqbs"]
Sep 10 13:38:43.248035 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:43.247995    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9217f8be-4e16-4dea-96d9-e46e7c21fe23" path="/var/lib/kubelet/pods/9217f8be-4e16-4dea-96d9-e46e7c21fe23/volumes"
Sep 10 13:38:44.305538 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:44.305504    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:38:44.432679 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:44.432639    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:44.432804 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:44.432695    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:44.445528 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:44.445505    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:45.140952 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:45.140925    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:38:46.379044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:46.379007    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:38:49.097874 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:49.097839    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:38:52.958406 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:38:52.958370    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:08.552063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:08.552033    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:08.658590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:08.658560    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:08.898641 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:08.898555    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:10.303442 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:10.303409    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:10.403200 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:10.403174    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:10.468835 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:10.468805    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:11.199782 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:11.199751    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:11.369484 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:11.369449    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:11.465435 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:11.465362    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:12.785924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:12.785893    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:14.553629 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:14.553590    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:39:19.710714 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:19.710631    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:19.810311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:19.810281    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:20.439499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:20.439469    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:25.036535 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.036502    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"]
Sep 10 13:39:25.037129 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.037061    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="599996d1-3cdb-431d-b6ad-dd1ba1d5e999" containerName="main"
Sep 10 13:39:25.037129 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.037082    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="599996d1-3cdb-431d-b6ad-dd1ba1d5e999" containerName="main"
Sep 10 13:39:25.037129 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.037102    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9217f8be-4e16-4dea-96d9-e46e7c21fe23" containerName="main"
Sep 10 13:39:25.037129 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.037110    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9217f8be-4e16-4dea-96d9-e46e7c21fe23" containerName="main"
Sep 10 13:39:25.037343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.037219    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9217f8be-4e16-4dea-96d9-e46e7c21fe23" containerName="main"
Sep 10 13:39:25.037343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.037233    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="599996d1-3cdb-431d-b6ad-dd1ba1d5e999" containerName="main"
Sep 10 13:39:25.041809 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.041788    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.044979 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.044959    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"kube-root-ca.crt\""
Sep 10 13:39:25.045102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.044966    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"openshift-service-ca.crt\""
Sep 10 13:39:25.046105 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.046086    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"default-dockercfg-g5jcc\""
Sep 10 13:39:25.046375 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.046362    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"sub-test-model-7739cb-kserve-self-signed-certs\""
Sep 10 13:39:25.052761 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.052738    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"]
Sep 10 13:39:25.070954 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.070927    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:25.108371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.108343    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-home\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.108477 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.108375    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-dshm\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.108477 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.108404    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-tmp-dir\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.108477 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.108459    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psc9f\" (UniqueName: \"kubernetes.io/projected/aeb2a600-ca4b-45a9-b537-d73748556de7-kube-api-access-psc9f\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.108653 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.108479    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-model-cache\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.108653 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.108496    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb2a600-ca4b-45a9-b537-d73748556de7-tls-certs\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.208975 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.208948    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-tmp-dir\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209094 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.208989    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psc9f\" (UniqueName: \"kubernetes.io/projected/aeb2a600-ca4b-45a9-b537-d73748556de7-kube-api-access-psc9f\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209094 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.209008    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-model-cache\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209094 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.209030    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb2a600-ca4b-45a9-b537-d73748556de7-tls-certs\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209264 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.209117    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-home\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209264 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.209141    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-dshm\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209400 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.209380    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-tmp-dir\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.209397    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-model-cache\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.209450 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.209424    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-home\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.211267 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.211246    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-dshm\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.211518 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.211502    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb2a600-ca4b-45a9-b537-d73748556de7-tls-certs\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.218004 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.217987    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psc9f\" (UniqueName: \"kubernetes.io/projected/aeb2a600-ca4b-45a9-b537-d73748556de7-kube-api-access-psc9f\") pod \"sub-test-model-7739cb-kserve-6679c74b77-h8sv2\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.351055 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.350988    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:25.625160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:25.625069    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"]
Sep 10 13:39:25.628185 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:39:25.628159    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaeb2a600_ca4b_45a9_b537_d73748556de7.slice/crio-1a36eb10d0b308094aee3f0d839162c8075773997cfcd335208754d8be90bac8 WatchSource:0}: Error finding container 1a36eb10d0b308094aee3f0d839162c8075773997cfcd335208754d8be90bac8: Status 404 returned error can't find the container with id 1a36eb10d0b308094aee3f0d839162c8075773997cfcd335208754d8be90bac8
Sep 10 13:39:26.328490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:26.328455    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2" event={"ID":"aeb2a600-ca4b-45a9-b537-d73748556de7","Type":"ContainerStarted","Data":"0d9d062b69d94a678357991e1799b149fd26199bdeca6f3638dab8ea21cdde84"}
Sep 10 13:39:26.328490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:26.328501    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2" event={"ID":"aeb2a600-ca4b-45a9-b537-d73748556de7","Type":"ContainerStarted","Data":"1a36eb10d0b308094aee3f0d839162c8075773997cfcd335208754d8be90bac8"}
Sep 10 13:39:26.353093 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:26.353050    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2" podStartSLOduration=1.353034636 podStartE2EDuration="1.353034636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:39:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:39:26.351857928 +0000 UTC m=+1331.563051184" watchObservedRunningTime="2026-09-10 13:39:26.353034636 +0000 UTC m=+1331.564227887"
Sep 10 13:39:27.551486 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:27.551429    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:27.675085 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:27.675053    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:27.765654 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:27.765626    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:28.955068 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:28.955032    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:29.057478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:29.057443    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:31.894371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:31.894329    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:39:35.352088 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:35.352054    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:35.352088 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:35.352094    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:35.364513 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:35.364481    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:35.382497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:35.382472    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:39:38.434163 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:38.434081    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:38.688023 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:38.687913    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:45.098978 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:45.098942    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:39:57.468896 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:57.468849    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:39:59.720829 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:39:59.720757    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:02.778643 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:02.778606    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:40:06.315016 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.314983    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:06.437190 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.437154    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:06.536017 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.535996    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:06.761534 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.761462    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7836-3a90a8"]
Sep 10 13:40:06.764704 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.764680    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:06.769144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.769121    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tqk99\""
Sep 10 13:40:06.773732 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.773709    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7836-3a90a8"]
Sep 10 13:40:06.808189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.808170    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:06.954733 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:06.954701    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpsqd\" (UniqueName: \"kubernetes.io/projected/f4c7dbd0-433d-4120-bfc4-49520b94515f-kube-api-access-bpsqd\") pod \"test-curl-7836-3a90a8\" (UID: \"f4c7dbd0-433d-4120-bfc4-49520b94515f\") " pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:07.055767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.055735    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpsqd\" (UniqueName: \"kubernetes.io/projected/f4c7dbd0-433d-4120-bfc4-49520b94515f-kube-api-access-bpsqd\") pod \"test-curl-7836-3a90a8\" (UID: \"f4c7dbd0-433d-4120-bfc4-49520b94515f\") " pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:07.064578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.064557    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpsqd\" (UniqueName: \"kubernetes.io/projected/f4c7dbd0-433d-4120-bfc4-49520b94515f-kube-api-access-bpsqd\") pod \"test-curl-7836-3a90a8\" (UID: \"f4c7dbd0-433d-4120-bfc4-49520b94515f\") " pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:07.074414 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.074385    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:07.248613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.248583    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:07.360038 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.359962    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:07.366009 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.365982    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7836-3a90a8"]
Sep 10 13:40:07.369275 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:07.369252    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4c7dbd0_433d_4120_bfc4_49520b94515f.slice/crio-0827d3993321ee039c06b2b7538b9f5e07afddc2311d36b538906554c5fc6035 WatchSource:0}: Error finding container 0827d3993321ee039c06b2b7538b9f5e07afddc2311d36b538906554c5fc6035: Status 404 returned error can't find the container with id 0827d3993321ee039c06b2b7538b9f5e07afddc2311d36b538906554c5fc6035
Sep 10 13:40:07.515689 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.515651    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7836-3a90a8" event={"ID":"f4c7dbd0-433d-4120-bfc4-49520b94515f","Type":"ContainerStarted","Data":"8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25"}
Sep 10 13:40:07.515795 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.515707    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7836-3a90a8" event={"ID":"f4c7dbd0-433d-4120-bfc4-49520b94515f","Type":"ContainerStarted","Data":"0827d3993321ee039c06b2b7538b9f5e07afddc2311d36b538906554c5fc6035"}
Sep 10 13:40:07.545100 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:07.545055    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7836-3a90a8" podStartSLOduration=1.545040954 podStartE2EDuration="1.545040954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:07.543201024 +0000 UTC m=+1372.754394282" watchObservedRunningTime="2026-09-10 13:40:07.545040954 +0000 UTC m=+1372.756234211"
Sep 10 13:40:08.309445 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.309368    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7836-3a90a8"]
Sep 10 13:40:08.314688 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.314648    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7836-3a90a8"]
Sep 10 13:40:08.451126 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.451098    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7836-9eceb3"]
Sep 10 13:40:08.451488 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.451475    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c7dbd0-433d-4120-bfc4-49520b94515f" containerName="test-curl-7836-3a90a8"
Sep 10 13:40:08.451532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.451490    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c7dbd0-433d-4120-bfc4-49520b94515f" containerName="test-curl-7836-3a90a8"
Sep 10 13:40:08.451569 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.451562    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4c7dbd0-433d-4120-bfc4-49520b94515f" containerName="test-curl-7836-3a90a8"
Sep 10 13:40:08.454640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.454624    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:08.463517 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.463481    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7836-9eceb3"]
Sep 10 13:40:08.520083 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:08.520060    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7836-3a90a8" container="test-curl-7836-3a90a8"
Sep 10 13:40:08.567915 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.567857    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssx7k\" (UniqueName: \"kubernetes.io/projected/433b0ebf-6621-4a3d-937f-90c010ec18d2-kube-api-access-ssx7k\") pod \"test-curl-7836-9eceb3\" (UID: \"433b0ebf-6621-4a3d-937f-90c010ec18d2\") " pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:08.668465 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.668438    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssx7k\" (UniqueName: \"kubernetes.io/projected/433b0ebf-6621-4a3d-937f-90c010ec18d2-kube-api-access-ssx7k\") pod \"test-curl-7836-9eceb3\" (UID: \"433b0ebf-6621-4a3d-937f-90c010ec18d2\") " pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:08.678770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.678745    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssx7k\" (UniqueName: \"kubernetes.io/projected/433b0ebf-6621-4a3d-937f-90c010ec18d2-kube-api-access-ssx7k\") pod \"test-curl-7836-9eceb3\" (UID: \"433b0ebf-6621-4a3d-937f-90c010ec18d2\") " pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:08.765169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:08.765146    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:09.045420 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:09.045397    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7836-9eceb3"]
Sep 10 13:40:09.047219 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:09.047190    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod433b0ebf_6621_4a3d_937f_90c010ec18d2.slice/crio-aa19c3242757735e1c5e5a215ccf6e7436cca16cc5e0bc0f74d66e034ccdfd44 WatchSource:0}: Error finding container aa19c3242757735e1c5e5a215ccf6e7436cca16cc5e0bc0f74d66e034ccdfd44: Status 404 returned error can't find the container with id aa19c3242757735e1c5e5a215ccf6e7436cca16cc5e0bc0f74d66e034ccdfd44
Sep 10 13:40:09.527453 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:09.527419    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7836-9eceb3" event={"ID":"433b0ebf-6621-4a3d-937f-90c010ec18d2","Type":"ContainerStarted","Data":"59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110"}
Sep 10 13:40:09.527887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:09.527461    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7836-9eceb3" event={"ID":"433b0ebf-6621-4a3d-937f-90c010ec18d2","Type":"ContainerStarted","Data":"aa19c3242757735e1c5e5a215ccf6e7436cca16cc5e0bc0f74d66e034ccdfd44"}
Sep 10 13:40:09.527887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:09.527570    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7836-3a90a8" podUID="f4c7dbd0-433d-4120-bfc4-49520b94515f" containerName="test-curl-7836-3a90a8" containerID="cri-o://8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25" gracePeriod=2
Sep 10 13:40:09.551260 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:09.551216    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7836-9eceb3" podStartSLOduration=1.551201561 podStartE2EDuration="1.551201561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:09.550280866 +0000 UTC m=+1374.761474124" watchObservedRunningTime="2026-09-10 13:40:09.551201561 +0000 UTC m=+1374.762394865"
Sep 10 13:40:10.321227 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:10.321197    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7836-9eceb3"]
Sep 10 13:40:10.325470 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:10.325443    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7836-9eceb3"]
Sep 10 13:40:10.532521 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:10.532478    2551 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7836-9eceb3" container="test-curl-7836-9eceb3"
Sep 10 13:40:10.715225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:10.715200    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:10.831222 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:10.831121    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:11.537317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:11.537286    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7836-9eceb3" podUID="433b0ebf-6621-4a3d-937f-90c010ec18d2" containerName="test-curl-7836-9eceb3" containerID="cri-o://59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110" gracePeriod=2
Sep 10 13:40:11.715089 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:11.715066    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:11.889807 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:11.889729    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:11.896856 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:11.896835    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpsqd\" (UniqueName: \"kubernetes.io/projected/f4c7dbd0-433d-4120-bfc4-49520b94515f-kube-api-access-bpsqd\") pod \"f4c7dbd0-433d-4120-bfc4-49520b94515f\" (UID: \"f4c7dbd0-433d-4120-bfc4-49520b94515f\") "
Sep 10 13:40:11.898809 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:11.898780    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4c7dbd0-433d-4120-bfc4-49520b94515f-kube-api-access-bpsqd" (OuterVolumeSpecName: "kube-api-access-bpsqd") pod "f4c7dbd0-433d-4120-bfc4-49520b94515f" (UID: "f4c7dbd0-433d-4120-bfc4-49520b94515f"). InnerVolumeSpecName "kube-api-access-bpsqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:40:11.991031 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:11.991003    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:12.004397 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.004363    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpsqd\" (UniqueName: \"kubernetes.io/projected/f4c7dbd0-433d-4120-bfc4-49520b94515f-kube-api-access-bpsqd\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:12.503625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.503587    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:12.523965 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.523937    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"]
Sep 10 13:40:12.524295 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.524281    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="433b0ebf-6621-4a3d-937f-90c010ec18d2" containerName="test-curl-7836-9eceb3"
Sep 10 13:40:12.524339 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.524297    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="433b0ebf-6621-4a3d-937f-90c010ec18d2" containerName="test-curl-7836-9eceb3"
Sep 10 13:40:12.524378 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.524370    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="433b0ebf-6621-4a3d-937f-90c010ec18d2" containerName="test-curl-7836-9eceb3"
Sep 10 13:40:12.527408 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.527389    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.530186 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.530170    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-559bdf-dockercfg-2d8fw\""
Sep 10 13:40:12.530564 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.530548    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-559bdf\""
Sep 10 13:40:12.541598 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.541576    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"]
Sep 10 13:40:12.543756 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.543735    2551 generic.go:356] "Generic (PLEG): container finished" podID="f4c7dbd0-433d-4120-bfc4-49520b94515f" containerID="8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25" exitCode=137
Sep 10 13:40:12.543851 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.543844    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:12.545024 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.544998    2551 scope.go:117] "RemoveContainer" containerID="8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25"
Sep 10 13:40:12.545206 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.545070    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.549423 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.549401    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"]
Sep 10 13:40:12.558904 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.558867    2551 scope.go:117] "RemoveContainer" containerID="8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25"
Sep 10 13:40:12.559358 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:12.559338    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25\": container with ID starting with 8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25 not found: ID does not exist" containerID="8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25"
Sep 10 13:40:12.559496 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.559473    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25"} err="failed to get container status \"8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25\": rpc error: code = NotFound desc = could not find container \"8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25\": container with ID starting with 8dcd5b50b34cb012479c21ab1bdc7618169fdba8b818f2b5f052c1e8c567ab25 not found: ID does not exist"
Sep 10 13:40:12.561637 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.561618    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"]
Sep 10 13:40:12.575398 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:12.575369    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7836-3a90a8\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="f4c7dbd0-433d-4120-bfc4-49520b94515f" pod="opendatahub/test-curl-7836-3a90a8"
Sep 10 13:40:12.610605 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.610578    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f402f1-91a4-44f1-9911-11cf43f83a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.610710 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.610635    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfvr9\" (UniqueName: \"kubernetes.io/projected/54f402f1-91a4-44f1-9911-11cf43f83a58-kube-api-access-zfvr9\") pod \"payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.610754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.610726    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c9117e-9876-419a-9ed0-40047652c5d1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") " pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.610754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.610742    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2br2b\" (UniqueName: \"kubernetes.io/projected/00c9117e-9876-419a-9ed0-40047652c5d1-kube-api-access-2br2b\") pod \"payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") " pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.711443 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.711415    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c9117e-9876-419a-9ed0-40047652c5d1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") " pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.711443 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.711446    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2br2b\" (UniqueName: \"kubernetes.io/projected/00c9117e-9876-419a-9ed0-40047652c5d1-kube-api-access-2br2b\") pod \"payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") " pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.711594 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.711491    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f402f1-91a4-44f1-9911-11cf43f83a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.711594 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.711555    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfvr9\" (UniqueName: \"kubernetes.io/projected/54f402f1-91a4-44f1-9911-11cf43f83a58-kube-api-access-zfvr9\") pod \"payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.712155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.712136    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c9117e-9876-419a-9ed0-40047652c5d1-plugins-config-volume\") pod \"payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") " pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.712239 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.712220    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f402f1-91a4-44f1-9911-11cf43f83a58-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.733189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.733160    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2br2b\" (UniqueName: \"kubernetes.io/projected/00c9117e-9876-419a-9ed0-40047652c5d1-kube-api-access-2br2b\") pod \"payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") " pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.734631 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.734613    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfvr9\" (UniqueName: \"kubernetes.io/projected/54f402f1-91a4-44f1-9911-11cf43f83a58-kube-api-access-zfvr9\") pod \"payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.837137 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.837111    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:12.861191 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.861162    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:12.991024 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:12.990972    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:13.111250 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.111226    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"]
Sep 10 13:40:13.112575 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:13.112547    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f402f1_91a4_44f1_9911_11cf43f83a58.slice/crio-7a2632ea83a309125b224b6da95bbf60eb3b357ed49aabff202b6149dbe4cded WatchSource:0}: Error finding container 7a2632ea83a309125b224b6da95bbf60eb3b357ed49aabff202b6149dbe4cded: Status 404 returned error can't find the container with id 7a2632ea83a309125b224b6da95bbf60eb3b357ed49aabff202b6149dbe4cded
Sep 10 13:40:13.144367 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.144345    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"]
Sep 10 13:40:13.145708 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:13.145685    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c9117e_9876_419a_9ed0_40047652c5d1.slice/crio-a5697b4a523220b372db7e2188cddb13edb8ad080d2db6ad3d78b3b062341be4 WatchSource:0}: Error finding container a5697b4a523220b372db7e2188cddb13edb8ad080d2db6ad3d78b3b062341be4: Status 404 returned error can't find the container with id a5697b4a523220b372db7e2188cddb13edb8ad080d2db6ad3d78b3b062341be4
Sep 10 13:40:13.248976 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.248946    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4c7dbd0-433d-4120-bfc4-49520b94515f" path="/var/lib/kubelet/pods/f4c7dbd0-433d-4120-bfc4-49520b94515f/volumes"
Sep 10 13:40:13.551694 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.551642    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx" event={"ID":"54f402f1-91a4-44f1-9911-11cf43f83a58","Type":"ContainerStarted","Data":"3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640"}
Sep 10 13:40:13.552024 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.551713    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx" event={"ID":"54f402f1-91a4-44f1-9911-11cf43f83a58","Type":"ContainerStarted","Data":"7a2632ea83a309125b224b6da95bbf60eb3b357ed49aabff202b6149dbe4cded"}
Sep 10 13:40:13.552024 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.551746    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:13.553163 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.553143    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv" event={"ID":"00c9117e-9876-419a-9ed0-40047652c5d1","Type":"ContainerStarted","Data":"910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6"}
Sep 10 13:40:13.553270 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.553181    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv" event={"ID":"00c9117e-9876-419a-9ed0-40047652c5d1","Type":"ContainerStarted","Data":"a5697b4a523220b372db7e2188cddb13edb8ad080d2db6ad3d78b3b062341be4"}
Sep 10 13:40:13.553270 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.553254    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:13.584382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.584344    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx" podStartSLOduration=1.584330985 podStartE2EDuration="1.584330985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:13.584013039 +0000 UTC m=+1378.795206298" watchObservedRunningTime="2026-09-10 13:40:13.584330985 +0000 UTC m=+1378.795524243"
Sep 10 13:40:13.624952 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.624905    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv" podStartSLOduration=1.6248897260000001 podStartE2EDuration="1.624889726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:13.623497049 +0000 UTC m=+1378.834690308" watchObservedRunningTime="2026-09-10 13:40:13.624889726 +0000 UTC m=+1378.836083071"
Sep 10 13:40:13.687868 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.687838    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:13.720991 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.720963    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssx7k\" (UniqueName: \"kubernetes.io/projected/433b0ebf-6621-4a3d-937f-90c010ec18d2-kube-api-access-ssx7k\") pod \"433b0ebf-6621-4a3d-937f-90c010ec18d2\" (UID: \"433b0ebf-6621-4a3d-937f-90c010ec18d2\") "
Sep 10 13:40:13.722743 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.722709    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/433b0ebf-6621-4a3d-937f-90c010ec18d2-kube-api-access-ssx7k" (OuterVolumeSpecName: "kube-api-access-ssx7k") pod "433b0ebf-6621-4a3d-937f-90c010ec18d2" (UID: "433b0ebf-6621-4a3d-937f-90c010ec18d2"). InnerVolumeSpecName "kube-api-access-ssx7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:40:13.822074 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.822011    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssx7k\" (UniqueName: \"kubernetes.io/projected/433b0ebf-6621-4a3d-937f-90c010ec18d2-kube-api-access-ssx7k\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:13.920169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.920141    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"]
Sep 10 13:40:13.935005 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:13.934979    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:14.558962 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:14.558927    2551 generic.go:356] "Generic (PLEG): container finished" podID="433b0ebf-6621-4a3d-937f-90c010ec18d2" containerID="59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110" exitCode=137
Sep 10 13:40:14.559365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:14.558974    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:14.559365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:14.559070    2551 scope.go:117] "RemoveContainer" containerID="59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110"
Sep 10 13:40:14.568321 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:14.568303    2551 scope.go:117] "RemoveContainer" containerID="59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110"
Sep 10 13:40:14.568611 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:14.568594    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110\": container with ID starting with 59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110 not found: ID does not exist" containerID="59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110"
Sep 10 13:40:14.568689 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:14.568623    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110"} err="failed to get container status \"59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110\": rpc error: code = NotFound desc = could not find container \"59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110\": container with ID starting with 59ddd180f41e6c66b2319bf59461566173f767ff312034d1c8a607129a632110 not found: ID does not exist"
Sep 10 13:40:14.570002 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:14.569978    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7836-9eceb3\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="433b0ebf-6621-4a3d-937f-90c010ec18d2" pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:15.248181 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:15.248147    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7836-9eceb3\" is forbidden: User \"system:node:ip-10-0-137-9.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-9.ec2.internal' and this object" podUID="433b0ebf-6621-4a3d-937f-90c010ec18d2" pod="opendatahub/test-curl-7836-9eceb3"
Sep 10 13:40:15.249548 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:15.249525    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="433b0ebf-6621-4a3d-937f-90c010ec18d2" path="/var/lib/kubelet/pods/433b0ebf-6621-4a3d-937f-90c010ec18d2/volumes"
Sep 10 13:40:15.416478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:15.416448    2551 scope.go:117] "RemoveContainer" containerID="87eb749be0ad681313fb0f7b87c7985a8dc1d35db7e068a448bee30528eabe1a"
Sep 10 13:40:15.501848 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:15.501783    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"]
Sep 10 13:40:15.549900 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:15.549871    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:17.096266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:17.096232    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:40:22.292225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.292192    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"]
Sep 10 13:40:22.297338 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.297319    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.304556 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.304534    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7657ef\"/\"rate-test-model-7657ef-kserve-self-signed-certs\""
Sep 10 13:40:22.332039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.332016    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"]
Sep 10 13:40:22.350994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.350973    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:22.387436 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.387413    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-model-cache\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.387522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.387439    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tls-certs\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.387522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.387456    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvjxp\" (UniqueName: \"kubernetes.io/projected/f36cfc7b-fd4c-4756-8956-d52f1fda8207-kube-api-access-vvjxp\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.387592 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.387544    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-home\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.387592 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.387565    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-dshm\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.387685 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.387619    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tmp-dir\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488122 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488099    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-home\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488124    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-dshm\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488153    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tmp-dir\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488182    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-model-cache\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488196    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tls-certs\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488211 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488211    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvjxp\" (UniqueName: \"kubernetes.io/projected/f36cfc7b-fd4c-4756-8956-d52f1fda8207-kube-api-access-vvjxp\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488587 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488566    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tmp-dir\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488587 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488578    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-home\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.488734 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.488616    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-model-cache\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.490260 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.490241    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-dshm\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.490593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.490574    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tls-certs\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.498105 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.498085    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvjxp\" (UniqueName: \"kubernetes.io/projected/f36cfc7b-fd4c-4756-8956-d52f1fda8207-kube-api-access-vvjxp\") pod \"rate-test-model-7657ef-kserve-64bdc755-dwd7k\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") " pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.606454 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.606394    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:22.880311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:22.880242    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"]
Sep 10 13:40:22.883518 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:22.883492    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf36cfc7b_fd4c_4756_8956_d52f1fda8207.slice/crio-e1aab839f000e9efc5b97a3a6a58c73c6601a9810bb47046e4bccf2f6bef50a8 WatchSource:0}: Error finding container e1aab839f000e9efc5b97a3a6a58c73c6601a9810bb47046e4bccf2f6bef50a8: Status 404 returned error can't find the container with id e1aab839f000e9efc5b97a3a6a58c73c6601a9810bb47046e4bccf2f6bef50a8
Sep 10 13:40:23.608688 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:23.608639    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k" event={"ID":"f36cfc7b-fd4c-4756-8956-d52f1fda8207","Type":"ContainerStarted","Data":"937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493"}
Sep 10 13:40:23.609015 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:23.608691    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k" event={"ID":"f36cfc7b-fd4c-4756-8956-d52f1fda8207","Type":"ContainerStarted","Data":"e1aab839f000e9efc5b97a3a6a58c73c6601a9810bb47046e4bccf2f6bef50a8"}
Sep 10 13:40:23.632389 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:23.632347    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k" podStartSLOduration=1.632332677 podStartE2EDuration="1.632332677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:23.631156344 +0000 UTC m=+1388.842349602" watchObservedRunningTime="2026-09-10 13:40:23.632332677 +0000 UTC m=+1388.843525933"
Sep 10 13:40:24.560845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:24.560814    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:40:24.561822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:24.561802    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:40:25.839569 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:25.839526    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:26.281739 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:26.281714    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:28.939373 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:28.939330    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh" podUID="8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" containerName="main" containerID="cri-o://c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6" gracePeriod=45
Sep 10 13:40:29.181263 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.181243    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:40:29.245032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.244968    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tls-certs\") pod \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") "
Sep 10 13:40:29.245032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245027    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-model-cache\") pod \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") "
Sep 10 13:40:29.245229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245055    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-home\") pod \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") "
Sep 10 13:40:29.245229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245125    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfhx2\" (UniqueName: \"kubernetes.io/projected/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-kube-api-access-cfhx2\") pod \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") "
Sep 10 13:40:29.245229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245149    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-dshm\") pod \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") "
Sep 10 13:40:29.245229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245208    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tmp-dir\") pod \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\" (UID: \"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e\") "
Sep 10 13:40:29.245447 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245285    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-model-cache" (OuterVolumeSpecName: "model-cache") pod "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" (UID: "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:29.245507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245448    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-home" (OuterVolumeSpecName: "home") pod "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" (UID: "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:29.245507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245487    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:29.245597 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.245516    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" (UID: "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:29.247356 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.247328    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" (UID: "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:40:29.247356 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.247342    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-dshm" (OuterVolumeSpecName: "dshm") pod "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" (UID: "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:29.247505 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.247389    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-kube-api-access-cfhx2" (OuterVolumeSpecName: "kube-api-access-cfhx2") pod "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" (UID: "8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e"). InnerVolumeSpecName "kube-api-access-cfhx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:40:29.346535 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.346511    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfhx2\" (UniqueName: \"kubernetes.io/projected/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-kube-api-access-cfhx2\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:29.346535 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.346532    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:29.346535 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.346542    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:29.346797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.346550    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:29.346797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.346561    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:29.638558 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.638524    2551 generic.go:356] "Generic (PLEG): container finished" podID="8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" containerID="c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6" exitCode=0
Sep 10 13:40:29.638730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.638562    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"
Sep 10 13:40:29.638730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.638612    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh" event={"ID":"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e","Type":"ContainerDied","Data":"c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6"}
Sep 10 13:40:29.638730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.638647    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh" event={"ID":"8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e","Type":"ContainerDied","Data":"0258c358ac7f5f10612b6e45b2ca405909aea866b81fac7684fcc34babf8cef7"}
Sep 10 13:40:29.638730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.638678    2551 scope.go:117] "RemoveContainer" containerID="c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6"
Sep 10 13:40:29.647352 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.647337    2551 scope.go:117] "RemoveContainer" containerID="c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6"
Sep 10 13:40:29.647602 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:40:29.647582    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6\": container with ID starting with c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6 not found: ID does not exist" containerID="c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6"
Sep 10 13:40:29.647658 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.647609    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6"} err="failed to get container status \"c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6\": rpc error: code = NotFound desc = could not find container \"c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6\": container with ID starting with c8f1dca19ad49e522769e1b6e288c774e1e84302a9b42b4879374016cc8e79e6 not found: ID does not exist"
Sep 10 13:40:29.657617 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.657592    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"]
Sep 10 13:40:29.663015 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:29.662997    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/sub-test-model-7657ef-kserve-66f45d5579-p26gh"]
Sep 10 13:40:30.521003 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.520955    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2" podUID="aeb2a600-ca4b-45a9-b537-d73748556de7" containerName="main" containerID="cri-o://0d9d062b69d94a678357991e1799b149fd26199bdeca6f3638dab8ea21cdde84" gracePeriod=45
Sep 10 13:40:30.647335 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.647304    2551 generic.go:356] "Generic (PLEG): container finished" podID="aeb2a600-ca4b-45a9-b537-d73748556de7" containerID="0d9d062b69d94a678357991e1799b149fd26199bdeca6f3638dab8ea21cdde84" exitCode=0
Sep 10 13:40:30.647447 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.647374    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2" event={"ID":"aeb2a600-ca4b-45a9-b537-d73748556de7","Type":"ContainerDied","Data":"0d9d062b69d94a678357991e1799b149fd26199bdeca6f3638dab8ea21cdde84"}
Sep 10 13:40:30.761190 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.761168    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:40:30.860322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860249    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psc9f\" (UniqueName: \"kubernetes.io/projected/aeb2a600-ca4b-45a9-b537-d73748556de7-kube-api-access-psc9f\") pod \"aeb2a600-ca4b-45a9-b537-d73748556de7\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") "
Sep 10 13:40:30.860322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860288    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-home\") pod \"aeb2a600-ca4b-45a9-b537-d73748556de7\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") "
Sep 10 13:40:30.860322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860317    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-model-cache\") pod \"aeb2a600-ca4b-45a9-b537-d73748556de7\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") "
Sep 10 13:40:30.860543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860427    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-tmp-dir\") pod \"aeb2a600-ca4b-45a9-b537-d73748556de7\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") "
Sep 10 13:40:30.860543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860483    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb2a600-ca4b-45a9-b537-d73748556de7-tls-certs\") pod \"aeb2a600-ca4b-45a9-b537-d73748556de7\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") "
Sep 10 13:40:30.860543 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860525    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-dshm\") pod \"aeb2a600-ca4b-45a9-b537-d73748556de7\" (UID: \"aeb2a600-ca4b-45a9-b537-d73748556de7\") "
Sep 10 13:40:30.860712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860552    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-home" (OuterVolumeSpecName: "home") pod "aeb2a600-ca4b-45a9-b537-d73748556de7" (UID: "aeb2a600-ca4b-45a9-b537-d73748556de7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:30.860712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860562    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-model-cache" (OuterVolumeSpecName: "model-cache") pod "aeb2a600-ca4b-45a9-b537-d73748556de7" (UID: "aeb2a600-ca4b-45a9-b537-d73748556de7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:30.860712 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860700    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aeb2a600-ca4b-45a9-b537-d73748556de7" (UID: "aeb2a600-ca4b-45a9-b537-d73748556de7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:30.860888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860826    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:30.860888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860846    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:30.860888 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.860860    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:30.862397 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.862376    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb2a600-ca4b-45a9-b537-d73748556de7-kube-api-access-psc9f" (OuterVolumeSpecName: "kube-api-access-psc9f") pod "aeb2a600-ca4b-45a9-b537-d73748556de7" (UID: "aeb2a600-ca4b-45a9-b537-d73748556de7"). InnerVolumeSpecName "kube-api-access-psc9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:40:30.862495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.862477    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-dshm" (OuterVolumeSpecName: "dshm") pod "aeb2a600-ca4b-45a9-b537-d73748556de7" (UID: "aeb2a600-ca4b-45a9-b537-d73748556de7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:40:30.862597 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.862578    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb2a600-ca4b-45a9-b537-d73748556de7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aeb2a600-ca4b-45a9-b537-d73748556de7" (UID: "aeb2a600-ca4b-45a9-b537-d73748556de7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:40:30.961434 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.961411    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psc9f\" (UniqueName: \"kubernetes.io/projected/aeb2a600-ca4b-45a9-b537-d73748556de7-kube-api-access-psc9f\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:30.961434 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.961431    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb2a600-ca4b-45a9-b537-d73748556de7-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:30.961572 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:30.961440    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb2a600-ca4b-45a9-b537-d73748556de7-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:40:31.248204 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:31.248129    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" path="/var/lib/kubelet/pods/8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e/volumes"
Sep 10 13:40:31.654170 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:31.654145    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"
Sep 10 13:40:31.654613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:31.654178    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2" event={"ID":"aeb2a600-ca4b-45a9-b537-d73748556de7","Type":"ContainerDied","Data":"1a36eb10d0b308094aee3f0d839162c8075773997cfcd335208754d8be90bac8"}
Sep 10 13:40:31.654613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:31.654234    2551 scope.go:117] "RemoveContainer" containerID="0d9d062b69d94a678357991e1799b149fd26199bdeca6f3638dab8ea21cdde84"
Sep 10 13:40:31.672489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:31.672462    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"]
Sep 10 13:40:31.677264 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:31.677242    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/sub-test-model-7739cb-kserve-6679c74b77-h8sv2"]
Sep 10 13:40:32.607292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:32.607261    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:32.607485 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:32.607305    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:32.619329 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:32.619306    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:32.671120 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:32.671091    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:40:33.248129 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:33.248095    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb2a600-ca4b-45a9-b537-d73748556de7" path="/var/lib/kubelet/pods/aeb2a600-ca4b-45a9-b537-d73748556de7/volumes"
Sep 10 13:40:35.715082 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:35.715054    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:36.259502 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:36.259465    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:36.526941 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:36.526907    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:40:36.623478 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:36.623444    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:36.979167 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:36.979092    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:37.074933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:37.074905    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:39.410606 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:39.410575    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:39.504894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:39.504867    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:40.623526 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:40.623496    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:40.689591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:40.689563    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:40.979132 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:40.979052    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:41.165047 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.165014    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:41.264203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.264125    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:41.284189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.284160    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"]
Sep 10 13:40:41.284540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.284527    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" containerName="main"
Sep 10 13:40:41.284584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.284542    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" containerName="main"
Sep 10 13:40:41.284584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.284572    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb2a600-ca4b-45a9-b537-d73748556de7" containerName="main"
Sep 10 13:40:41.284584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.284578    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb2a600-ca4b-45a9-b537-d73748556de7" containerName="main"
Sep 10 13:40:41.284693 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.284629    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb2a600-ca4b-45a9-b537-d73748556de7" containerName="main"
Sep 10 13:40:41.284693 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.284642    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e0cd0fa-33ec-46a6-9cb7-e7fd2c23671e" containerName="main"
Sep 10 13:40:41.289090 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.289075    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.292252 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.292235    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-fa4d53\""
Sep 10 13:40:41.292513 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.292500    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-fa4d53-dockercfg-qsdqh\""
Sep 10 13:40:41.298889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.298871    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"]
Sep 10 13:40:41.318175 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.318148    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"]
Sep 10 13:40:41.321383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.321369    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.334887 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.334864    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"]
Sep 10 13:40:41.349566 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.349540    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cdf780-d7dd-43f0-a844-8e090333fd90-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.349703 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.349597    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x784c\" (UniqueName: \"kubernetes.io/projected/a1cdf780-d7dd-43f0-a844-8e090333fd90-kube-api-access-x784c\") pod \"payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.450504 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.450481    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805774c1-d52c-4a0f-b9b6-5b57242d4c16-plugins-config-volume\") pod \"payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") " pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.450624 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.450530    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvp6s\" (UniqueName: \"kubernetes.io/projected/805774c1-d52c-4a0f-b9b6-5b57242d4c16-kube-api-access-lvp6s\") pod \"payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") " pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.450624 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.450564    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cdf780-d7dd-43f0-a844-8e090333fd90-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.450778 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.450623    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x784c\" (UniqueName: \"kubernetes.io/projected/a1cdf780-d7dd-43f0-a844-8e090333fd90-kube-api-access-x784c\") pod \"payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.451188 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.451172    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cdf780-d7dd-43f0-a844-8e090333fd90-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.461954 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.461935    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x784c\" (UniqueName: \"kubernetes.io/projected/a1cdf780-d7dd-43f0-a844-8e090333fd90-kube-api-access-x784c\") pod \"payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.551885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.551860    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805774c1-d52c-4a0f-b9b6-5b57242d4c16-plugins-config-volume\") pod \"payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") " pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.551972 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.551912    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvp6s\" (UniqueName: \"kubernetes.io/projected/805774c1-d52c-4a0f-b9b6-5b57242d4c16-kube-api-access-lvp6s\") pod \"payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") " pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.552481 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.552463    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805774c1-d52c-4a0f-b9b6-5b57242d4c16-plugins-config-volume\") pod \"payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") " pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.562215 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.562185    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvp6s\" (UniqueName: \"kubernetes.io/projected/805774c1-d52c-4a0f-b9b6-5b57242d4c16-kube-api-access-lvp6s\") pod \"payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") " pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.599266 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.599244    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:41.630276 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.630245    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:41.875580 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.875555    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"]
Sep 10 13:40:41.877741 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:41.877715    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cdf780_d7dd_43f0_a844_8e090333fd90.slice/crio-44c52163326193a58efd02bbc4b65ba2e7768dee2f84078187409307804d9777 WatchSource:0}: Error finding container 44c52163326193a58efd02bbc4b65ba2e7768dee2f84078187409307804d9777: Status 404 returned error can't find the container with id 44c52163326193a58efd02bbc4b65ba2e7768dee2f84078187409307804d9777
Sep 10 13:40:41.912108 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:41.912067    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"]
Sep 10 13:40:41.913845 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:41.913823    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod805774c1_d52c_4a0f_b9b6_5b57242d4c16.slice/crio-df47355dc2464078b3f8861e96725b940f6a1e7449d8fe61748f6616e1d54f09 WatchSource:0}: Error finding container df47355dc2464078b3f8861e96725b940f6a1e7449d8fe61748f6616e1d54f09: Status 404 returned error can't find the container with id df47355dc2464078b3f8861e96725b940f6a1e7449d8fe61748f6616e1d54f09
Sep 10 13:40:42.716134 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.716095    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf" event={"ID":"805774c1-d52c-4a0f-b9b6-5b57242d4c16","Type":"ContainerStarted","Data":"bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c"}
Sep 10 13:40:42.716608 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.716306    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:42.716608 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.716341    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf" event={"ID":"805774c1-d52c-4a0f-b9b6-5b57242d4c16","Type":"ContainerStarted","Data":"df47355dc2464078b3f8861e96725b940f6a1e7449d8fe61748f6616e1d54f09"}
Sep 10 13:40:42.717861 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.717830    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t" event={"ID":"a1cdf780-d7dd-43f0-a844-8e090333fd90","Type":"ContainerStarted","Data":"ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858"}
Sep 10 13:40:42.717978 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.717889    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t" event={"ID":"a1cdf780-d7dd-43f0-a844-8e090333fd90","Type":"ContainerStarted","Data":"44c52163326193a58efd02bbc4b65ba2e7768dee2f84078187409307804d9777"}
Sep 10 13:40:42.718044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.717969    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:42.739879 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.739842    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf" podStartSLOduration=1.739829477 podStartE2EDuration="1.739829477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:42.737434257 +0000 UTC m=+1407.948627513" watchObservedRunningTime="2026-09-10 13:40:42.739829477 +0000 UTC m=+1407.951022734"
Sep 10 13:40:42.763345 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:42.763304    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t" podStartSLOduration=1.763293033 podStartE2EDuration="1.763293033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:42.761020761 +0000 UTC m=+1407.972214030" watchObservedRunningTime="2026-09-10 13:40:42.763293033 +0000 UTC m=+1407.974486290"
Sep 10 13:40:48.495091 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:48.495053    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:40:52.074754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:52.074723    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:53.725417 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:53.725389    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:40:53.725797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:53.725462    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:40:56.029625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.029580    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"]
Sep 10 13:40:56.035518 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.035494    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.039042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.039021    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"kube-root-ca.crt\""
Sep 10 13:40:56.040386 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.040367    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"openshift-service-ca.crt\""
Sep 10 13:40:56.040459 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.040444    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"default-dockercfg-g5jcc\""
Sep 10 13:40:56.040523 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.040444    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-7739cb\"/\"rate-test-model-7739cb-kserve-self-signed-certs\""
Sep 10 13:40:56.046311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.046292    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"]
Sep 10 13:40:56.062878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.062856    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:40:56.169789 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.169761    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-dshm\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.169919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.169796    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-home\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.169919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.169814    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tls-certs\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.169919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.169882    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tmp-dir\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.170027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.169927    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-model-cache\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.170027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.169968    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s8kc\" (UniqueName: \"kubernetes.io/projected/8eee55c8-bd31-40d7-bdda-2ac70023aa36-kube-api-access-7s8kc\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.270579 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.270552    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-dshm\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.270724 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.270585    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-home\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.270724 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.270599    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tls-certs\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.270835 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.270721    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tmp-dir\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.270835 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.270750    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-model-cache\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.270942 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.270897    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s8kc\" (UniqueName: \"kubernetes.io/projected/8eee55c8-bd31-40d7-bdda-2ac70023aa36-kube-api-access-7s8kc\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.271032 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.271002    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-home\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.271152 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.271056    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-model-cache\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.271152 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.271136    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tmp-dir\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.272787 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.272769    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-dshm\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.273201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.273183    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tls-certs\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.282731 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.282653    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s8kc\" (UniqueName: \"kubernetes.io/projected/8eee55c8-bd31-40d7-bdda-2ac70023aa36-kube-api-access-7s8kc\") pod \"rate-test-model-7739cb-kserve-95c66868c-sd2gr\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") " pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.345692 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.345655    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:40:56.621366 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.621344    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"]
Sep 10 13:40:56.623235 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:40:56.623207    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8eee55c8_bd31_40d7_bdda_2ac70023aa36.slice/crio-87c043eb2dbd77f534c57044b8ed0080a3885d2bf704cf7317755e5c76f4cf5d WatchSource:0}: Error finding container 87c043eb2dbd77f534c57044b8ed0080a3885d2bf704cf7317755e5c76f4cf5d: Status 404 returned error can't find the container with id 87c043eb2dbd77f534c57044b8ed0080a3885d2bf704cf7317755e5c76f4cf5d
Sep 10 13:40:56.790592 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.790551    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr" event={"ID":"8eee55c8-bd31-40d7-bdda-2ac70023aa36","Type":"ContainerStarted","Data":"407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558"}
Sep 10 13:40:56.790592 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.790595    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr" event={"ID":"8eee55c8-bd31-40d7-bdda-2ac70023aa36","Type":"ContainerStarted","Data":"87c043eb2dbd77f534c57044b8ed0080a3885d2bf704cf7317755e5c76f4cf5d"}
Sep 10 13:40:56.813167 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:40:56.813120    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr" podStartSLOduration=0.813106067 podStartE2EDuration="813.106067ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:40:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:40:56.812183506 +0000 UTC m=+1422.023376765" watchObservedRunningTime="2026-09-10 13:40:56.813106067 +0000 UTC m=+1422.024299323"
Sep 10 13:41:01.626802 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:01.626762    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:01.806964 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:01.806927    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:41:04.568494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:04.568462    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:06.346870 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:06.346832    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:41:06.347308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:06.346885    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:41:06.358956 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:06.358933    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:41:06.851200 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:06.851172    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:41:08.179924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:08.179846    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:15.442806 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:15.442774    2551 scope.go:117] "RemoveContainer" containerID="2c4a88627f9c88990a1272587c39b23f25e0a158832e7f3b22daccec22cd2bd5"
Sep 10 13:41:21.957221 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:21.957190    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:23.865639 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:23.865603    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"]
Sep 10 13:41:23.866111 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:23.865911    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv" podUID="00c9117e-9876-419a-9ed0-40047652c5d1" containerName="payload-processing" containerID="cri-o://910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6" gracePeriod=30
Sep 10 13:41:23.870951 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:23.870923    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:23.878853 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:23.878831    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"]
Sep 10 13:41:23.879038 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:23.879005    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx" podUID="54f402f1-91a4-44f1-9911-11cf43f83a58" containerName="payload-pre-processing" containerID="cri-o://3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640" gracePeriod=30
Sep 10 13:41:23.979018 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:23.978990    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:24.116178 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.116112    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:24.134321 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.134296    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:41:24.165276 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.165250    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:41:24.197416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.197392    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f402f1-91a4-44f1-9911-11cf43f83a58-plugins-config-volume\") pod \"54f402f1-91a4-44f1-9911-11cf43f83a58\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") "
Sep 10 13:41:24.197522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.197457    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c9117e-9876-419a-9ed0-40047652c5d1-plugins-config-volume\") pod \"00c9117e-9876-419a-9ed0-40047652c5d1\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") "
Sep 10 13:41:24.197522 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.197490    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2br2b\" (UniqueName: \"kubernetes.io/projected/00c9117e-9876-419a-9ed0-40047652c5d1-kube-api-access-2br2b\") pod \"00c9117e-9876-419a-9ed0-40047652c5d1\" (UID: \"00c9117e-9876-419a-9ed0-40047652c5d1\") "
Sep 10 13:41:24.197612 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.197534    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfvr9\" (UniqueName: \"kubernetes.io/projected/54f402f1-91a4-44f1-9911-11cf43f83a58-kube-api-access-zfvr9\") pod \"54f402f1-91a4-44f1-9911-11cf43f83a58\" (UID: \"54f402f1-91a4-44f1-9911-11cf43f83a58\") "
Sep 10 13:41:24.197853 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.197830    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54f402f1-91a4-44f1-9911-11cf43f83a58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54f402f1-91a4-44f1-9911-11cf43f83a58" (UID: "54f402f1-91a4-44f1-9911-11cf43f83a58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:41:24.197897 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.197834    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00c9117e-9876-419a-9ed0-40047652c5d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00c9117e-9876-419a-9ed0-40047652c5d1" (UID: "00c9117e-9876-419a-9ed0-40047652c5d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:41:24.199540 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.199511    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54f402f1-91a4-44f1-9911-11cf43f83a58-kube-api-access-zfvr9" (OuterVolumeSpecName: "kube-api-access-zfvr9") pod "54f402f1-91a4-44f1-9911-11cf43f83a58" (UID: "54f402f1-91a4-44f1-9911-11cf43f83a58"). InnerVolumeSpecName "kube-api-access-zfvr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:24.199615 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.199552    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00c9117e-9876-419a-9ed0-40047652c5d1-kube-api-access-2br2b" (OuterVolumeSpecName: "kube-api-access-2br2b") pod "00c9117e-9876-419a-9ed0-40047652c5d1" (UID: "00c9117e-9876-419a-9ed0-40047652c5d1"). InnerVolumeSpecName "kube-api-access-2br2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:24.208169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.208141    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:24.298360 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.298333    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00c9117e-9876-419a-9ed0-40047652c5d1-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:24.298360 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.298356    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2br2b\" (UniqueName: \"kubernetes.io/projected/00c9117e-9876-419a-9ed0-40047652c5d1-kube-api-access-2br2b\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:24.298499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.298367    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfvr9\" (UniqueName: \"kubernetes.io/projected/54f402f1-91a4-44f1-9911-11cf43f83a58-kube-api-access-zfvr9\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:24.298499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.298380    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54f402f1-91a4-44f1-9911-11cf43f83a58-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:24.629625 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.629599    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:24.928123 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.928092    2551 generic.go:356] "Generic (PLEG): container finished" podID="54f402f1-91a4-44f1-9911-11cf43f83a58" containerID="3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640" exitCode=0
Sep 10 13:41:24.928582 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.928137    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"
Sep 10 13:41:24.928582 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.928176    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx" event={"ID":"54f402f1-91a4-44f1-9911-11cf43f83a58","Type":"ContainerDied","Data":"3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640"}
Sep 10 13:41:24.928582 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.928229    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx" event={"ID":"54f402f1-91a4-44f1-9911-11cf43f83a58","Type":"ContainerDied","Data":"7a2632ea83a309125b224b6da95bbf60eb3b357ed49aabff202b6149dbe4cded"}
Sep 10 13:41:24.928582 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.928251    2551 scope.go:117] "RemoveContainer" containerID="3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640"
Sep 10 13:41:24.929613 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.929591    2551 generic.go:356] "Generic (PLEG): container finished" podID="00c9117e-9876-419a-9ed0-40047652c5d1" containerID="910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6" exitCode=0
Sep 10 13:41:24.929714 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.929630    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"
Sep 10 13:41:24.929714 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.929642    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv" event={"ID":"00c9117e-9876-419a-9ed0-40047652c5d1","Type":"ContainerDied","Data":"910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6"}
Sep 10 13:41:24.929714 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.929693    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv" event={"ID":"00c9117e-9876-419a-9ed0-40047652c5d1","Type":"ContainerDied","Data":"a5697b4a523220b372db7e2188cddb13edb8ad080d2db6ad3d78b3b062341be4"}
Sep 10 13:41:24.938021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.937811    2551 scope.go:117] "RemoveContainer" containerID="3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640"
Sep 10 13:41:24.938171 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:41:24.938147    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640\": container with ID starting with 3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640 not found: ID does not exist" containerID="3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640"
Sep 10 13:41:24.938236 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.938185    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640"} err="failed to get container status \"3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640\": rpc error: code = NotFound desc = could not find container \"3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640\": container with ID starting with 3dd5f12345df47b02df08284af28c6d5d7d42421707349024ea6dd3d2cf32640 not found: ID does not exist"
Sep 10 13:41:24.938236 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.938217    2551 scope.go:117] "RemoveContainer" containerID="910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6"
Sep 10 13:41:24.945969 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.945947    2551 scope.go:117] "RemoveContainer" containerID="910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6"
Sep 10 13:41:24.946246 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:41:24.946224    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6\": container with ID starting with 910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6 not found: ID does not exist" containerID="910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6"
Sep 10 13:41:24.946297 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.946256    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6"} err="failed to get container status \"910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6\": rpc error: code = NotFound desc = could not find container \"910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6\": container with ID starting with 910e56bc3f66c3d0d23e664c22e9f2f589c76d44796939ef0498e21c60d3edd6 not found: ID does not exist"
Sep 10 13:41:24.962889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.962869    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"]
Sep 10 13:41:24.971094 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.971074    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-559bdf-5f4d844f9c-rkrnv"]
Sep 10 13:41:24.994208 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.994184    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"]
Sep 10 13:41:24.999278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:24.999256    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-559bdf-5d6c866454-zzklx"]
Sep 10 13:41:25.249514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:25.249444    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00c9117e-9876-419a-9ed0-40047652c5d1" path="/var/lib/kubelet/pods/00c9117e-9876-419a-9ed0-40047652c5d1/volumes"
Sep 10 13:41:25.249810 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:25.249794    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54f402f1-91a4-44f1-9911-11cf43f83a58" path="/var/lib/kubelet/pods/54f402f1-91a4-44f1-9911-11cf43f83a58/volumes"
Sep 10 13:41:25.411928 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:25.411901    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:41:32.833748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:32.833717    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:32.957183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:32.957151    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:33.253077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:33.252992    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:33.890985 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:33.890949    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:43.917903 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:43.917865    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:41:48.649225 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:48.649189    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"]
Sep 10 13:41:48.692297 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:48.692261    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:48.940274 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:48.940194    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:49.558046 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:49.558008    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"]
Sep 10 13:41:50.049935 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.049899    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"]
Sep 10 13:41:50.050400 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.050171    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf" podUID="805774c1-d52c-4a0f-b9b6-5b57242d4c16" containerName="payload-processing" containerID="cri-o://bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c" gracePeriod=30
Sep 10 13:41:50.060589 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.060562    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"]
Sep 10 13:41:50.060817 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.060794    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t" podUID="a1cdf780-d7dd-43f0-a844-8e090333fd90" containerName="payload-pre-processing" containerID="cri-o://ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858" gracePeriod=30
Sep 10 13:41:50.306072 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.306008    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:41:50.335840 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.335817    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:41:50.436739 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.436708    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvp6s\" (UniqueName: \"kubernetes.io/projected/805774c1-d52c-4a0f-b9b6-5b57242d4c16-kube-api-access-lvp6s\") pod \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") "
Sep 10 13:41:50.436900 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.436789    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cdf780-d7dd-43f0-a844-8e090333fd90-plugins-config-volume\") pod \"a1cdf780-d7dd-43f0-a844-8e090333fd90\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") "
Sep 10 13:41:50.436900 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.436860    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x784c\" (UniqueName: \"kubernetes.io/projected/a1cdf780-d7dd-43f0-a844-8e090333fd90-kube-api-access-x784c\") pod \"a1cdf780-d7dd-43f0-a844-8e090333fd90\" (UID: \"a1cdf780-d7dd-43f0-a844-8e090333fd90\") "
Sep 10 13:41:50.436989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.436938    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805774c1-d52c-4a0f-b9b6-5b57242d4c16-plugins-config-volume\") pod \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\" (UID: \"805774c1-d52c-4a0f-b9b6-5b57242d4c16\") "
Sep 10 13:41:50.437301 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.437268    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1cdf780-d7dd-43f0-a844-8e090333fd90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1cdf780-d7dd-43f0-a844-8e090333fd90" (UID: "a1cdf780-d7dd-43f0-a844-8e090333fd90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:41:50.437426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.437310    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/805774c1-d52c-4a0f-b9b6-5b57242d4c16-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "805774c1-d52c-4a0f-b9b6-5b57242d4c16" (UID: "805774c1-d52c-4a0f-b9b6-5b57242d4c16"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:41:50.438915 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.438893    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805774c1-d52c-4a0f-b9b6-5b57242d4c16-kube-api-access-lvp6s" (OuterVolumeSpecName: "kube-api-access-lvp6s") pod "805774c1-d52c-4a0f-b9b6-5b57242d4c16" (UID: "805774c1-d52c-4a0f-b9b6-5b57242d4c16"). InnerVolumeSpecName "kube-api-access-lvp6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:50.439079 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.439053    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1cdf780-d7dd-43f0-a844-8e090333fd90-kube-api-access-x784c" (OuterVolumeSpecName: "kube-api-access-x784c") pod "a1cdf780-d7dd-43f0-a844-8e090333fd90" (UID: "a1cdf780-d7dd-43f0-a844-8e090333fd90"). InnerVolumeSpecName "kube-api-access-x784c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:41:50.538764 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.538731    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvp6s\" (UniqueName: \"kubernetes.io/projected/805774c1-d52c-4a0f-b9b6-5b57242d4c16-kube-api-access-lvp6s\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:50.538764 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.538757    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1cdf780-d7dd-43f0-a844-8e090333fd90-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:50.538764 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.538766    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x784c\" (UniqueName: \"kubernetes.io/projected/a1cdf780-d7dd-43f0-a844-8e090333fd90-kube-api-access-x784c\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:50.538764 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.538775    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/805774c1-d52c-4a0f-b9b6-5b57242d4c16-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:41:50.546961 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.546930    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:50.649039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.648949    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:50.850751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.850716    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:50.923349 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:50.923263    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:51.061757 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.061724    2551 generic.go:356] "Generic (PLEG): container finished" podID="805774c1-d52c-4a0f-b9b6-5b57242d4c16" containerID="bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c" exitCode=0
Sep 10 13:41:51.062203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.061774    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"
Sep 10 13:41:51.062203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.061810    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf" event={"ID":"805774c1-d52c-4a0f-b9b6-5b57242d4c16","Type":"ContainerDied","Data":"bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c"}
Sep 10 13:41:51.062203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.061862    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf" event={"ID":"805774c1-d52c-4a0f-b9b6-5b57242d4c16","Type":"ContainerDied","Data":"df47355dc2464078b3f8861e96725b940f6a1e7449d8fe61748f6616e1d54f09"}
Sep 10 13:41:51.062203 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.061886    2551 scope.go:117] "RemoveContainer" containerID="bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c"
Sep 10 13:41:51.063383 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.063361    2551 generic.go:356] "Generic (PLEG): container finished" podID="a1cdf780-d7dd-43f0-a844-8e090333fd90" containerID="ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858" exitCode=0
Sep 10 13:41:51.063545 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.063391    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t" event={"ID":"a1cdf780-d7dd-43f0-a844-8e090333fd90","Type":"ContainerDied","Data":"ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858"}
Sep 10 13:41:51.063545 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.063425    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"
Sep 10 13:41:51.063545 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.063453    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t" event={"ID":"a1cdf780-d7dd-43f0-a844-8e090333fd90","Type":"ContainerDied","Data":"44c52163326193a58efd02bbc4b65ba2e7768dee2f84078187409307804d9777"}
Sep 10 13:41:51.072606 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.072513    2551 scope.go:117] "RemoveContainer" containerID="bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c"
Sep 10 13:41:51.072865 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:41:51.072847    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c\": container with ID starting with bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c not found: ID does not exist" containerID="bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c"
Sep 10 13:41:51.072933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.072873    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c"} err="failed to get container status \"bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c\": rpc error: code = NotFound desc = could not find container \"bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c\": container with ID starting with bcaed167814e445b1424eb5f62b0214465c33b3f1e4fc4f88f043e1594367a7c not found: ID does not exist"
Sep 10 13:41:51.072933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.072889    2551 scope.go:117] "RemoveContainer" containerID="ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858"
Sep 10 13:41:51.083583 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.083559    2551 scope.go:117] "RemoveContainer" containerID="ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858"
Sep 10 13:41:51.083949 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:41:51.083917    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858\": container with ID starting with ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858 not found: ID does not exist" containerID="ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858"
Sep 10 13:41:51.084056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.083956    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858"} err="failed to get container status \"ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858\": rpc error: code = NotFound desc = could not find container \"ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858\": container with ID starting with ad45e95cb75eda7ac05cd32bd8f26a32b219f468969becf8ae1bd48e52bd9858 not found: ID does not exist"
Sep 10 13:41:51.096382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.096356    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"]
Sep 10 13:41:51.099502 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.099480    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-fa4d53-6fb5475b4c-5wcrf"]
Sep 10 13:41:51.132893 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.132867    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"]
Sep 10 13:41:51.140104 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.140078    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-fa4d53-69fc6466-48j8t"]
Sep 10 13:41:51.248800 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.248721    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805774c1-d52c-4a0f-b9b6-5b57242d4c16" path="/var/lib/kubelet/pods/805774c1-d52c-4a0f-b9b6-5b57242d4c16/volumes"
Sep 10 13:41:51.249090 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:51.249078    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1cdf780-d7dd-43f0-a844-8e090333fd90" path="/var/lib/kubelet/pods/a1cdf780-d7dd-43f0-a844-8e090333fd90/volumes"
Sep 10 13:41:53.822786 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:53.822743    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:53.948102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:53.948063    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:55.047487 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.047458    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:55.136575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.136540    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:55.239018 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.238990    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:55.745675 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.745627    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:55.853852 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.853815    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:55.985898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.985864    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"]
Sep 10 13:41:55.986308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986291    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00c9117e-9876-419a-9ed0-40047652c5d1" containerName="payload-processing"
Sep 10 13:41:55.986403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986311    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="00c9117e-9876-419a-9ed0-40047652c5d1" containerName="payload-processing"
Sep 10 13:41:55.986403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986332    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805774c1-d52c-4a0f-b9b6-5b57242d4c16" containerName="payload-processing"
Sep 10 13:41:55.986403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986341    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="805774c1-d52c-4a0f-b9b6-5b57242d4c16" containerName="payload-processing"
Sep 10 13:41:55.986403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986366    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1cdf780-d7dd-43f0-a844-8e090333fd90" containerName="payload-pre-processing"
Sep 10 13:41:55.986403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986375    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1cdf780-d7dd-43f0-a844-8e090333fd90" containerName="payload-pre-processing"
Sep 10 13:41:55.986403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986391    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54f402f1-91a4-44f1-9911-11cf43f83a58" containerName="payload-pre-processing"
Sep 10 13:41:55.986403 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986399    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="54f402f1-91a4-44f1-9911-11cf43f83a58" containerName="payload-pre-processing"
Sep 10 13:41:55.986774 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986486    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="00c9117e-9876-419a-9ed0-40047652c5d1" containerName="payload-processing"
Sep 10 13:41:55.986774 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986505    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="805774c1-d52c-4a0f-b9b6-5b57242d4c16" containerName="payload-processing"
Sep 10 13:41:55.986774 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986516    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="54f402f1-91a4-44f1-9911-11cf43f83a58" containerName="payload-pre-processing"
Sep 10 13:41:55.986774 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.986525    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1cdf780-d7dd-43f0-a844-8e090333fd90" containerName="payload-pre-processing"
Sep 10 13:41:55.991630 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.991605    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:55.994656 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.994634    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-56b405\""
Sep 10 13:41:55.994971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:55.994953    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-56b405-dockercfg-6g72g\""
Sep 10 13:41:56.002536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.002511    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"]
Sep 10 13:41:56.066754 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.066723    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"]
Sep 10 13:41:56.076914 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.076868    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.086588 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.086563    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb2n2\" (UniqueName: \"kubernetes.io/projected/84a875d7-c911-4e32-9d3f-c308859cdd34-kube-api-access-pb2n2\") pod \"payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:56.086779 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.086601    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a875d7-c911-4e32-9d3f-c308859cdd34-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:56.092346 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.092319    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"]
Sep 10 13:41:56.187606 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.187568    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d192a663-422e-4a90-a620-7bfe159a9117-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.187846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.187623    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb2n2\" (UniqueName: \"kubernetes.io/projected/84a875d7-c911-4e32-9d3f-c308859cdd34-kube-api-access-pb2n2\") pod \"payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:56.187846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.187700    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a875d7-c911-4e32-9d3f-c308859cdd34-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:56.187846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.187780    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqg68\" (UniqueName: \"kubernetes.io/projected/d192a663-422e-4a90-a620-7bfe159a9117-kube-api-access-jqg68\") pod \"payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.188434 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.188408    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a875d7-c911-4e32-9d3f-c308859cdd34-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:56.198830 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.198808    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb2n2\" (UniqueName: \"kubernetes.io/projected/84a875d7-c911-4e32-9d3f-c308859cdd34-kube-api-access-pb2n2\") pod \"payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:56.288842 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.288811    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d192a663-422e-4a90-a620-7bfe159a9117-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.289037 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.288885    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqg68\" (UniqueName: \"kubernetes.io/projected/d192a663-422e-4a90-a620-7bfe159a9117-kube-api-access-jqg68\") pod \"payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.289569 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.289540    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d192a663-422e-4a90-a620-7bfe159a9117-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.298722 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.298695    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqg68\" (UniqueName: \"kubernetes.io/projected/d192a663-422e-4a90-a620-7bfe159a9117-kube-api-access-jqg68\") pod \"payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.302116 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.302100    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:56.387021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.386992    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:56.584069 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.584040    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"]
Sep 10 13:41:56.585113 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:41:56.585092    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84a875d7_c911_4e32_9d3f_c308859cdd34.slice/crio-38864bad59d94a05f36c1f26c62f1c998122f59c3f0bd9fb3f723444a3efd649 WatchSource:0}: Error finding container 38864bad59d94a05f36c1f26c62f1c998122f59c3f0bd9fb3f723444a3efd649: Status 404 returned error can't find the container with id 38864bad59d94a05f36c1f26c62f1c998122f59c3f0bd9fb3f723444a3efd649
Sep 10 13:41:56.671560 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:56.671527    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"]
Sep 10 13:41:56.674641 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:41:56.674614    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd192a663_422e_4a90_a620_7bfe159a9117.slice/crio-9cc3aec87a6b10c216d2ea144521fd4e0cf310c3f524d46b09160683007c32a3 WatchSource:0}: Error finding container 9cc3aec87a6b10c216d2ea144521fd4e0cf310c3f524d46b09160683007c32a3: Status 404 returned error can't find the container with id 9cc3aec87a6b10c216d2ea144521fd4e0cf310c3f524d46b09160683007c32a3
Sep 10 13:41:57.100344 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.100264    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v" event={"ID":"d192a663-422e-4a90-a620-7bfe159a9117","Type":"ContainerStarted","Data":"883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8"}
Sep 10 13:41:57.100344 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.100340    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v" event={"ID":"d192a663-422e-4a90-a620-7bfe159a9117","Type":"ContainerStarted","Data":"9cc3aec87a6b10c216d2ea144521fd4e0cf310c3f524d46b09160683007c32a3"}
Sep 10 13:41:57.100864 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.100369    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:41:57.102039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.102014    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp" event={"ID":"84a875d7-c911-4e32-9d3f-c308859cdd34","Type":"ContainerStarted","Data":"ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78"}
Sep 10 13:41:57.102193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.102058    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp" event={"ID":"84a875d7-c911-4e32-9d3f-c308859cdd34","Type":"ContainerStarted","Data":"38864bad59d94a05f36c1f26c62f1c998122f59c3f0bd9fb3f723444a3efd649"}
Sep 10 13:41:57.102193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.102125    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:41:57.139479 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.139424    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v" podStartSLOduration=1.13940722 podStartE2EDuration="1.13940722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:57.139158205 +0000 UTC m=+1482.350351463" watchObservedRunningTime="2026-09-10 13:41:57.13940722 +0000 UTC m=+1482.350600529"
Sep 10 13:41:57.173412 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:57.173354    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp" podStartSLOduration=2.173333618 podStartE2EDuration="2.173333618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:41:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:41:57.171104936 +0000 UTC m=+1482.382298205" watchObservedRunningTime="2026-09-10 13:41:57.173333618 +0000 UTC m=+1482.384526874"
Sep 10 13:41:58.561471 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:58.561430    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:58.664177 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:58.664142    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:58.870727 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:58.870637    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:41:59.451938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:41:59.451904    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:02.018919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:02.018883    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:42:03.669193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.669155    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k" podUID="f36cfc7b-fd4c-4756-8956-d52f1fda8207" containerName="main" containerID="cri-o://937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493" gracePeriod=45
Sep 10 13:42:03.916781 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.916755    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:42:03.955358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955255    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tls-certs\") pod \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") "
Sep 10 13:42:03.955358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955328    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-dshm\") pod \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") "
Sep 10 13:42:03.955593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955361    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-model-cache\") pod \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") "
Sep 10 13:42:03.955593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955386    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tmp-dir\") pod \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") "
Sep 10 13:42:03.955593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955438    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvjxp\" (UniqueName: \"kubernetes.io/projected/f36cfc7b-fd4c-4756-8956-d52f1fda8207-kube-api-access-vvjxp\") pod \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") "
Sep 10 13:42:03.955797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955628    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-model-cache" (OuterVolumeSpecName: "model-cache") pod "f36cfc7b-fd4c-4756-8956-d52f1fda8207" (UID: "f36cfc7b-fd4c-4756-8956-d52f1fda8207"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:03.955797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955732    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f36cfc7b-fd4c-4756-8956-d52f1fda8207" (UID: "f36cfc7b-fd4c-4756-8956-d52f1fda8207"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:03.955797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955467    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-home\") pod \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\" (UID: \"f36cfc7b-fd4c-4756-8956-d52f1fda8207\") "
Sep 10 13:42:03.955995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.955975    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-home" (OuterVolumeSpecName: "home") pod "f36cfc7b-fd4c-4756-8956-d52f1fda8207" (UID: "f36cfc7b-fd4c-4756-8956-d52f1fda8207"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:03.956363 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.956342    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:03.956363 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.956366    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:03.956541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.956378    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:03.958040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.958011    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-dshm" (OuterVolumeSpecName: "dshm") pod "f36cfc7b-fd4c-4756-8956-d52f1fda8207" (UID: "f36cfc7b-fd4c-4756-8956-d52f1fda8207"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:03.958155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.958124    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f36cfc7b-fd4c-4756-8956-d52f1fda8207" (UID: "f36cfc7b-fd4c-4756-8956-d52f1fda8207"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:42:03.958155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:03.958138    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36cfc7b-fd4c-4756-8956-d52f1fda8207-kube-api-access-vvjxp" (OuterVolumeSpecName: "kube-api-access-vvjxp") pod "f36cfc7b-fd4c-4756-8956-d52f1fda8207" (UID: "f36cfc7b-fd4c-4756-8956-d52f1fda8207"). InnerVolumeSpecName "kube-api-access-vvjxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:42:04.057091 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.057045    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f36cfc7b-fd4c-4756-8956-d52f1fda8207-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.057091 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.057089    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f36cfc7b-fd4c-4756-8956-d52f1fda8207-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.057289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.057104    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvjxp\" (UniqueName: \"kubernetes.io/projected/f36cfc7b-fd4c-4756-8956-d52f1fda8207-kube-api-access-vvjxp\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.147984 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.147950    2551 generic.go:356] "Generic (PLEG): container finished" podID="f36cfc7b-fd4c-4756-8956-d52f1fda8207" containerID="937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493" exitCode=0
Sep 10 13:42:04.148183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.147997    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"
Sep 10 13:42:04.148183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.148001    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k" event={"ID":"f36cfc7b-fd4c-4756-8956-d52f1fda8207","Type":"ContainerDied","Data":"937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493"}
Sep 10 13:42:04.148183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.148122    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k" event={"ID":"f36cfc7b-fd4c-4756-8956-d52f1fda8207","Type":"ContainerDied","Data":"e1aab839f000e9efc5b97a3a6a58c73c6601a9810bb47046e4bccf2f6bef50a8"}
Sep 10 13:42:04.148183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.148152    2551 scope.go:117] "RemoveContainer" containerID="937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493"
Sep 10 13:42:04.158517 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.158494    2551 scope.go:117] "RemoveContainer" containerID="937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493"
Sep 10 13:42:04.158866 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:42:04.158841    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493\": container with ID starting with 937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493 not found: ID does not exist" containerID="937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493"
Sep 10 13:42:04.158920 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.158876    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493"} err="failed to get container status \"937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493\": rpc error: code = NotFound desc = could not find container \"937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493\": container with ID starting with 937bf851d6872185fde7d521ddb54fe511d2168b649ea3e4c76e48058b485493 not found: ID does not exist"
Sep 10 13:42:04.174941 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.174899    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"]
Sep 10 13:42:04.183822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.183789    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7657ef/rate-test-model-7657ef-kserve-64bdc755-dwd7k"]
Sep 10 13:42:04.577085 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.577037    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr" podUID="8eee55c8-bd31-40d7-bdda-2ac70023aa36" containerName="main" containerID="cri-o://407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558" gracePeriod=45
Sep 10 13:42:04.826544 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.826522    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:42:04.863433 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.863361    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tls-certs\") pod \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") "
Sep 10 13:42:04.863556 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.863439    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-home\") pod \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") "
Sep 10 13:42:04.863696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.863659    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-home" (OuterVolumeSpecName: "home") pod "8eee55c8-bd31-40d7-bdda-2ac70023aa36" (UID: "8eee55c8-bd31-40d7-bdda-2ac70023aa36"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:04.863696 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.863689    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-dshm\") pod \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") "
Sep 10 13:42:04.863846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.863735    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-model-cache\") pod \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") "
Sep 10 13:42:04.863846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.863769    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tmp-dir\") pod \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") "
Sep 10 13:42:04.863846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.863823    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s8kc\" (UniqueName: \"kubernetes.io/projected/8eee55c8-bd31-40d7-bdda-2ac70023aa36-kube-api-access-7s8kc\") pod \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\" (UID: \"8eee55c8-bd31-40d7-bdda-2ac70023aa36\") "
Sep 10 13:42:04.864416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.864099    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.864416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.864177    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-model-cache" (OuterVolumeSpecName: "model-cache") pod "8eee55c8-bd31-40d7-bdda-2ac70023aa36" (UID: "8eee55c8-bd31-40d7-bdda-2ac70023aa36"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:04.864416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.864359    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8eee55c8-bd31-40d7-bdda-2ac70023aa36" (UID: "8eee55c8-bd31-40d7-bdda-2ac70023aa36"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:04.866282 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.866258    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-dshm" (OuterVolumeSpecName: "dshm") pod "8eee55c8-bd31-40d7-bdda-2ac70023aa36" (UID: "8eee55c8-bd31-40d7-bdda-2ac70023aa36"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:42:04.866511 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.866480    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8eee55c8-bd31-40d7-bdda-2ac70023aa36-kube-api-access-7s8kc" (OuterVolumeSpecName: "kube-api-access-7s8kc") pod "8eee55c8-bd31-40d7-bdda-2ac70023aa36" (UID: "8eee55c8-bd31-40d7-bdda-2ac70023aa36"). InnerVolumeSpecName "kube-api-access-7s8kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:42:04.866596 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.866515    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8eee55c8-bd31-40d7-bdda-2ac70023aa36" (UID: "8eee55c8-bd31-40d7-bdda-2ac70023aa36"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:42:04.964824 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.964800    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.964824 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.964823    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.964995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.964836    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s8kc\" (UniqueName: \"kubernetes.io/projected/8eee55c8-bd31-40d7-bdda-2ac70023aa36-kube-api-access-7s8kc\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.964995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.964848    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eee55c8-bd31-40d7-bdda-2ac70023aa36-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:04.964995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:04.964862    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eee55c8-bd31-40d7-bdda-2ac70023aa36-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:05.154416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.154344    2551 generic.go:356] "Generic (PLEG): container finished" podID="8eee55c8-bd31-40d7-bdda-2ac70023aa36" containerID="407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558" exitCode=0
Sep 10 13:42:05.154416 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.154381    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"
Sep 10 13:42:05.154607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.154415    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr" event={"ID":"8eee55c8-bd31-40d7-bdda-2ac70023aa36","Type":"ContainerDied","Data":"407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558"}
Sep 10 13:42:05.154607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.154453    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr" event={"ID":"8eee55c8-bd31-40d7-bdda-2ac70023aa36","Type":"ContainerDied","Data":"87c043eb2dbd77f534c57044b8ed0080a3885d2bf704cf7317755e5c76f4cf5d"}
Sep 10 13:42:05.154607 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.154479    2551 scope.go:117] "RemoveContainer" containerID="407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558"
Sep 10 13:42:05.163234 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.163212    2551 scope.go:117] "RemoveContainer" containerID="407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558"
Sep 10 13:42:05.163492 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:42:05.163476    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558\": container with ID starting with 407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558 not found: ID does not exist" containerID="407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558"
Sep 10 13:42:05.163534 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.163499    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558"} err="failed to get container status \"407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558\": rpc error: code = NotFound desc = could not find container \"407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558\": container with ID starting with 407cec396d0893d53c6d68c70edfe2bd4c744ea65673464355c0e6609d5d8558 not found: ID does not exist"
Sep 10 13:42:05.191947 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.191924    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"]
Sep 10 13:42:05.196527 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.196504    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-7739cb/rate-test-model-7739cb-kserve-95c66868c-sd2gr"]
Sep 10 13:42:05.248586 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.248562    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8eee55c8-bd31-40d7-bdda-2ac70023aa36" path="/var/lib/kubelet/pods/8eee55c8-bd31-40d7-bdda-2ac70023aa36/volumes"
Sep 10 13:42:05.248977 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:05.248963    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36cfc7b-fd4c-4756-8956-d52f1fda8207" path="/var/lib/kubelet/pods/f36cfc7b-fd4c-4756-8956-d52f1fda8207/volumes"
Sep 10 13:42:08.110364 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:08.110334    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:42:08.110844 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:08.110653    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:42:10.121779 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.121715    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:10.149413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.149386    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"]
Sep 10 13:42:10.149606 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.149580    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm" podUID="3f973059-047f-40a7-83a3-f1842d5d20ed" containerName="payload-processing" containerID="cri-o://884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4" gracePeriod=30
Sep 10 13:42:10.160836 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.160810    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"]
Sep 10 13:42:10.161042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.161019    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8" podUID="ad92952b-9156-4ac8-bd9b-01797d5e03f3" containerName="payload-pre-processing" containerID="cri-o://d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f" gracePeriod=30
Sep 10 13:42:10.260361 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.260333    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:10.346039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.345997    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:10.404576 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.404544    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:42:10.423636 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.423617    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:42:10.426321 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.426299    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:10.506293 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.506269    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfn8d\" (UniqueName: \"kubernetes.io/projected/ad92952b-9156-4ac8-bd9b-01797d5e03f3-kube-api-access-gfn8d\") pod \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") "
Sep 10 13:42:10.506476 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.506311    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f973059-047f-40a7-83a3-f1842d5d20ed-plugins-config-volume\") pod \"3f973059-047f-40a7-83a3-f1842d5d20ed\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") "
Sep 10 13:42:10.506476 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.506336    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d6gf\" (UniqueName: \"kubernetes.io/projected/3f973059-047f-40a7-83a3-f1842d5d20ed-kube-api-access-2d6gf\") pod \"3f973059-047f-40a7-83a3-f1842d5d20ed\" (UID: \"3f973059-047f-40a7-83a3-f1842d5d20ed\") "
Sep 10 13:42:10.506476 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.506408    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad92952b-9156-4ac8-bd9b-01797d5e03f3-plugins-config-volume\") pod \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\" (UID: \"ad92952b-9156-4ac8-bd9b-01797d5e03f3\") "
Sep 10 13:42:10.506758 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.506735    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f973059-047f-40a7-83a3-f1842d5d20ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f973059-047f-40a7-83a3-f1842d5d20ed" (UID: "3f973059-047f-40a7-83a3-f1842d5d20ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:42:10.506848 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.506813    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad92952b-9156-4ac8-bd9b-01797d5e03f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad92952b-9156-4ac8-bd9b-01797d5e03f3" (UID: "ad92952b-9156-4ac8-bd9b-01797d5e03f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:42:10.508426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.508405    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad92952b-9156-4ac8-bd9b-01797d5e03f3-kube-api-access-gfn8d" (OuterVolumeSpecName: "kube-api-access-gfn8d") pod "ad92952b-9156-4ac8-bd9b-01797d5e03f3" (UID: "ad92952b-9156-4ac8-bd9b-01797d5e03f3"). InnerVolumeSpecName "kube-api-access-gfn8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:42:10.508494 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.508428    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f973059-047f-40a7-83a3-f1842d5d20ed-kube-api-access-2d6gf" (OuterVolumeSpecName: "kube-api-access-2d6gf") pod "3f973059-047f-40a7-83a3-f1842d5d20ed" (UID: "3f973059-047f-40a7-83a3-f1842d5d20ed"). InnerVolumeSpecName "kube-api-access-2d6gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:42:10.607209 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.607179    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad92952b-9156-4ac8-bd9b-01797d5e03f3-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:10.607209 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.607205    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfn8d\" (UniqueName: \"kubernetes.io/projected/ad92952b-9156-4ac8-bd9b-01797d5e03f3-kube-api-access-gfn8d\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:10.607209 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.607214    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f973059-047f-40a7-83a3-f1842d5d20ed-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:10.607401 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.607223    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d6gf\" (UniqueName: \"kubernetes.io/projected/3f973059-047f-40a7-83a3-f1842d5d20ed-kube-api-access-2d6gf\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:10.815223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:10.815189    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:11.186275 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.186246    2551 generic.go:356] "Generic (PLEG): container finished" podID="3f973059-047f-40a7-83a3-f1842d5d20ed" containerID="884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4" exitCode=0
Sep 10 13:42:11.186652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.186287    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"
Sep 10 13:42:11.186652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.186331    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm" event={"ID":"3f973059-047f-40a7-83a3-f1842d5d20ed","Type":"ContainerDied","Data":"884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4"}
Sep 10 13:42:11.186652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.186382    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm" event={"ID":"3f973059-047f-40a7-83a3-f1842d5d20ed","Type":"ContainerDied","Data":"145cda0dc31620ee4d7357d4c9e16b3edd518b1ac545f3ebd49c588e04ecfbab"}
Sep 10 13:42:11.186652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.186406    2551 scope.go:117] "RemoveContainer" containerID="884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4"
Sep 10 13:42:11.187710 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.187630    2551 generic.go:356] "Generic (PLEG): container finished" podID="ad92952b-9156-4ac8-bd9b-01797d5e03f3" containerID="d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f" exitCode=0
Sep 10 13:42:11.187710 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.187698    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8" event={"ID":"ad92952b-9156-4ac8-bd9b-01797d5e03f3","Type":"ContainerDied","Data":"d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f"}
Sep 10 13:42:11.187857 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.187712    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"
Sep 10 13:42:11.187857 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.187749    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8" event={"ID":"ad92952b-9156-4ac8-bd9b-01797d5e03f3","Type":"ContainerDied","Data":"f4d5a12a69fd88fdf0eaac7c980642d9f5ab835d2c816e669c4371ee6a772470"}
Sep 10 13:42:11.203586 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.203557    2551 scope.go:117] "RemoveContainer" containerID="884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4"
Sep 10 13:42:11.204037 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:42:11.204009    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4\": container with ID starting with 884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4 not found: ID does not exist" containerID="884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4"
Sep 10 13:42:11.204144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.204043    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4"} err="failed to get container status \"884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4\": rpc error: code = NotFound desc = could not find container \"884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4\": container with ID starting with 884ad404e1ffdd15e798f457b7f23ca2ce6f3d3cbdb8106c46f587e1d2a883f4 not found: ID does not exist"
Sep 10 13:42:11.204144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.204060    2551 scope.go:117] "RemoveContainer" containerID="d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f"
Sep 10 13:42:11.211639 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.211613    2551 scope.go:117] "RemoveContainer" containerID="d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f"
Sep 10 13:42:11.211992 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:42:11.211970    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f\": container with ID starting with d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f not found: ID does not exist" containerID="d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f"
Sep 10 13:42:11.212063 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.212007    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f"} err="failed to get container status \"d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f\": rpc error: code = NotFound desc = could not find container \"d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f\": container with ID starting with d379c3382ff0e5e17836759e1773e10eb4368675fcb7bf7e951f9188f48d094f not found: ID does not exist"
Sep 10 13:42:11.216343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.216322    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"]
Sep 10 13:42:11.224498 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.224479    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-c165a4-544d679585-7hkgm"]
Sep 10 13:42:11.238276 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.238253    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"]
Sep 10 13:42:11.248379 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.248357    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f973059-047f-40a7-83a3-f1842d5d20ed" path="/var/lib/kubelet/pods/3f973059-047f-40a7-83a3-f1842d5d20ed/volumes"
Sep 10 13:42:11.248660 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:11.248644    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-c165a4-df49675c4-nl6q8"]
Sep 10 13:42:13.247922 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:13.247891    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad92952b-9156-4ac8-bd9b-01797d5e03f3" path="/var/lib/kubelet/pods/ad92952b-9156-4ac8-bd9b-01797d5e03f3/volumes"
Sep 10 13:42:15.256193 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:15.256164    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:42:15.259477 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:15.259447    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:42:16.456987 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:16.456954    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:18.215699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.215650    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:18.317386 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.317352    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:18.571268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.571241    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:18.934994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.934909    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:18.979300 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979273    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"]
Sep 10 13:42:18.979674 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979645    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f973059-047f-40a7-83a3-f1842d5d20ed" containerName="payload-processing"
Sep 10 13:42:18.979771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979677    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f973059-047f-40a7-83a3-f1842d5d20ed" containerName="payload-processing"
Sep 10 13:42:18.979771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979695    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad92952b-9156-4ac8-bd9b-01797d5e03f3" containerName="payload-pre-processing"
Sep 10 13:42:18.979771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979703    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad92952b-9156-4ac8-bd9b-01797d5e03f3" containerName="payload-pre-processing"
Sep 10 13:42:18.979771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979726    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8eee55c8-bd31-40d7-bdda-2ac70023aa36" containerName="main"
Sep 10 13:42:18.979771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979735    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8eee55c8-bd31-40d7-bdda-2ac70023aa36" containerName="main"
Sep 10 13:42:18.979771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979752    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36cfc7b-fd4c-4756-8956-d52f1fda8207" containerName="main"
Sep 10 13:42:18.979771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979759    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36cfc7b-fd4c-4756-8956-d52f1fda8207" containerName="main"
Sep 10 13:42:18.980107 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979855    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad92952b-9156-4ac8-bd9b-01797d5e03f3" containerName="payload-pre-processing"
Sep 10 13:42:18.980107 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979867    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8eee55c8-bd31-40d7-bdda-2ac70023aa36" containerName="main"
Sep 10 13:42:18.980107 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979878    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f973059-047f-40a7-83a3-f1842d5d20ed" containerName="payload-processing"
Sep 10 13:42:18.980107 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.979887    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f36cfc7b-fd4c-4756-8956-d52f1fda8207" containerName="main"
Sep 10 13:42:18.982909 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.982890    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"
Sep 10 13:42:18.986269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.986249    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:42:18.986889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.986872    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:42:18.995645 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:18.995625    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"]
Sep 10 13:42:19.073278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:19.073248    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55af0bf9-8ace-4bf9-8c42-97c97989bff2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79\" (UID: \"55af0bf9-8ace-4bf9-8c42-97c97989bff2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"
Sep 10 13:42:19.173886 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:19.173860    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55af0bf9-8ace-4bf9-8c42-97c97989bff2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79\" (UID: \"55af0bf9-8ace-4bf9-8c42-97c97989bff2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"
Sep 10 13:42:19.174426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:19.174406    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55af0bf9-8ace-4bf9-8c42-97c97989bff2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79\" (UID: \"55af0bf9-8ace-4bf9-8c42-97c97989bff2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"
Sep 10 13:42:19.213519 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:19.213461    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:19.293484 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:19.293462    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"
Sep 10 13:42:19.568194 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:19.568091    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"]
Sep 10 13:42:19.570900 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:42:19.570860    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55af0bf9_8ace_4bf9_8c42_97c97989bff2.slice/crio-aaccb584cf3dda3509815eb29b3c218322b94fe28b97611a6d03dbf0c2987190 WatchSource:0}: Error finding container aaccb584cf3dda3509815eb29b3c218322b94fe28b97611a6d03dbf0c2987190: Status 404 returned error can't find the container with id aaccb584cf3dda3509815eb29b3c218322b94fe28b97611a6d03dbf0c2987190
Sep 10 13:42:19.623786 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:19.623759    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:20.090168 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.090136    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"]
Sep 10 13:42:20.093566 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.093549    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.099971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.099691    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-08b85c-openshift-default-dockercfg-f9dfj\""
Sep 10 13:42:20.101954 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.101933    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 13:42:20.120131 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.120072    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"]
Sep 10 13:42:20.141234 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.141208    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:20.181738 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181708    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-podinfo\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.181820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181741    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-socket\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.181820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181764    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-certs\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.181820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181806    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-credential-socket\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.181924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181894    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-token\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.181924 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181911    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istiod-ca-cert\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.181992 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181961    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-data\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.182029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.181999    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54w29\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-kube-api-access-54w29\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.182029 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.182017    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-envoy\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.239491 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.239457    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79" event={"ID":"55af0bf9-8ace-4bf9-8c42-97c97989bff2","Type":"ContainerStarted","Data":"ddd0b692cdb26fcc7456cdd6b85ddbdf8d6bb1fb68892d0458309ce3d09112e8"}
Sep 10 13:42:20.239593 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.239527    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79" event={"ID":"55af0bf9-8ace-4bf9-8c42-97c97989bff2","Type":"ContainerStarted","Data":"aaccb584cf3dda3509815eb29b3c218322b94fe28b97611a6d03dbf0c2987190"}
Sep 10 13:42:20.259467 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.259415    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79" podStartSLOduration=2.259397526 podStartE2EDuration="2.259397526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:42:20.257572036 +0000 UTC m=+1505.468765307" watchObservedRunningTime="2026-09-10 13:42:20.259397526 +0000 UTC m=+1505.470590785"
Sep 10 13:42:20.283079 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283056    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-token\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283170 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283083    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istiod-ca-cert\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283170 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283102    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-data\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283170 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283128    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54w29\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-kube-api-access-54w29\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283282 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283240    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-envoy\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283325 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283309    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-podinfo\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283371 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283343    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-socket\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283424 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283384    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-certs\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283474 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283437    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-credential-socket\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283529 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283482    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-data\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283712    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-socket\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283798    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-certs\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283976 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283850    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-credential-socket\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.283976 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.283960    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istiod-ca-cert\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.285457 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.285435    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-envoy\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.285705 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.285686    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-podinfo\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.292628 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.292611    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54w29\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-kube-api-access-54w29\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.292765 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.292729    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-token\") pod \"e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") " pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.403486 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.403462    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:20.696530 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.696507    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"]
Sep 10 13:42:20.698306 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:42:20.698277    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9f8baf_6a23_46cc_93ce_1c899a25988f.slice/crio-7835889a8f0a23a60bd1bc50482d9cddcf8d29f94c58d29566902d533043f728 WatchSource:0}: Error finding container 7835889a8f0a23a60bd1bc50482d9cddcf8d29f94c58d29566902d533043f728: Status 404 returned error can't find the container with id 7835889a8f0a23a60bd1bc50482d9cddcf8d29f94c58d29566902d533043f728
Sep 10 13:42:20.700643 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.700627    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:42:20.747195 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:20.747168    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:21.245828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:21.245796    2551 generic.go:356] "Generic (PLEG): container finished" podID="55af0bf9-8ace-4bf9-8c42-97c97989bff2" containerID="ddd0b692cdb26fcc7456cdd6b85ddbdf8d6bb1fb68892d0458309ce3d09112e8" exitCode=0
Sep 10 13:42:21.248231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:21.248204    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79" event={"ID":"55af0bf9-8ace-4bf9-8c42-97c97989bff2","Type":"ContainerDied","Data":"ddd0b692cdb26fcc7456cdd6b85ddbdf8d6bb1fb68892d0458309ce3d09112e8"}
Sep 10 13:42:21.248345 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:21.248250    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" event={"ID":"4c9f8baf-6a23-46cc-93ce-1c899a25988f","Type":"ContainerStarted","Data":"7835889a8f0a23a60bd1bc50482d9cddcf8d29f94c58d29566902d533043f728"}
Sep 10 13:42:22.403453 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:22.403429    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"
Sep 10 13:42:22.503547 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:22.503481    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55af0bf9-8ace-4bf9-8c42-97c97989bff2-maas-api-service-ca\") pod \"55af0bf9-8ace-4bf9-8c42-97c97989bff2\" (UID: \"55af0bf9-8ace-4bf9-8c42-97c97989bff2\") "
Sep 10 13:42:22.503930 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:22.503902    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55af0bf9-8ace-4bf9-8c42-97c97989bff2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55af0bf9-8ace-4bf9-8c42-97c97989bff2" (UID: "55af0bf9-8ace-4bf9-8c42-97c97989bff2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:42:22.604684 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:22.604642    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55af0bf9-8ace-4bf9-8c42-97c97989bff2-maas-api-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:23.259999 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:23.259961    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79" event={"ID":"55af0bf9-8ace-4bf9-8c42-97c97989bff2","Type":"ContainerDied","Data":"aaccb584cf3dda3509815eb29b3c218322b94fe28b97611a6d03dbf0c2987190"}
Sep 10 13:42:23.260133 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:23.260012    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aaccb584cf3dda3509815eb29b3c218322b94fe28b97611a6d03dbf0c2987190"
Sep 10 13:42:23.260133 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:23.259970    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"
Sep 10 13:42:23.521838 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:23.521800    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:23.522202 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:23.521887    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:23.522202 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:23.521941    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:24.113010 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:24.112972    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:42:24.267077 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:24.267040    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" event={"ID":"4c9f8baf-6a23-46cc-93ce-1c899a25988f","Type":"ContainerStarted","Data":"6ff5e650a73d39d6337f29a26fc9c774401cfce8d86ff75dc4d87dcc9b9d0a2f"}
Sep 10 13:42:24.295544 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:24.295499    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" podStartSLOduration=1.474875256 podStartE2EDuration="4.295484942s" totalImagesPullingTime="2.820609686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:42:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:42:24.294981097 +0000 UTC m=+1509.506174355" watchObservedRunningTime="2026-09-10 13:42:24.295484942 +0000 UTC m=+1509.506678199"
Sep 10 13:42:24.404088 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:24.404018    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:24.413906 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:24.413878    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:25.271412 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:25.271377    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:25.272740 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:25.272718    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:26.433711 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:26.433682    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:26.585230 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:26.585199    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:27.722569 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:27.722526    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:27.828116 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:27.828082    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:27.935316 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:27.935285    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:28.494618 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.494584    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:28.589685 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.589641    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:28.722054 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.722027    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"]
Sep 10 13:42:28.722394 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.722381    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55af0bf9-8ace-4bf9-8c42-97c97989bff2" containerName="revoke-keys"
Sep 10 13:42:28.722439 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.722396    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="55af0bf9-8ace-4bf9-8c42-97c97989bff2" containerName="revoke-keys"
Sep 10 13:42:28.722471 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.722459    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="55af0bf9-8ace-4bf9-8c42-97c97989bff2" containerName="revoke-keys"
Sep 10 13:42:28.725834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.725817    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:28.729950 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.729923    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-08b85c\""
Sep 10 13:42:28.777076 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.777038    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-08b85c-dockercfg-k2rvh\""
Sep 10 13:42:28.780443 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.780420    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"]
Sep 10 13:42:28.857495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.857467    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4vsz\" (UniqueName: \"kubernetes.io/projected/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-kube-api-access-k4vsz\") pod \"payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:28.857631 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.857549    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:28.957918 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.957889    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:28.958040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.957934    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4vsz\" (UniqueName: \"kubernetes.io/projected/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-kube-api-access-k4vsz\") pod \"payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:28.958615 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:28.958591    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:29.008055 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.008026    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4vsz\" (UniqueName: \"kubernetes.io/projected/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-kube-api-access-k4vsz\") pod \"payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:29.035515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.035457    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:29.346570 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.346495    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"]
Sep 10 13:42:29.350111 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:42:29.350078    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba16e5ef_5e5f_48cb_b2a7_7ac3e9535269.slice/crio-8d490cabe10b858c4c7fc181a8fe34f75de3aeffb01763b6cf2cb8793280707b WatchSource:0}: Error finding container 8d490cabe10b858c4c7fc181a8fe34f75de3aeffb01763b6cf2cb8793280707b: Status 404 returned error can't find the container with id 8d490cabe10b858c4c7fc181a8fe34f75de3aeffb01763b6cf2cb8793280707b
Sep 10 13:42:29.381466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.381444    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:29.463770 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.463739    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"]
Sep 10 13:42:29.463997 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.463969    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb" podUID="02897304-5217-4697-8e96-9d0cf4ec83c1" containerName="payload-pre-processing" containerID="cri-o://bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9" gracePeriod=30
Sep 10 13:42:29.498578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.498510    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:29.593123 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.593086    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:29.702498 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.702469    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:29.711191 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.711167    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:42:29.740643 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.740616    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"]
Sep 10 13:42:29.751846 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.751825    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-e65037-zsq79"]
Sep 10 13:42:29.764348 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.764329    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8455z\" (UniqueName: \"kubernetes.io/projected/02897304-5217-4697-8e96-9d0cf4ec83c1-kube-api-access-8455z\") pod \"02897304-5217-4697-8e96-9d0cf4ec83c1\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") "
Sep 10 13:42:29.764445 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.764400    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02897304-5217-4697-8e96-9d0cf4ec83c1-plugins-config-volume\") pod \"02897304-5217-4697-8e96-9d0cf4ec83c1\" (UID: \"02897304-5217-4697-8e96-9d0cf4ec83c1\") "
Sep 10 13:42:29.764753 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.764730    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02897304-5217-4697-8e96-9d0cf4ec83c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "02897304-5217-4697-8e96-9d0cf4ec83c1" (UID: "02897304-5217-4697-8e96-9d0cf4ec83c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:42:29.766426 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.766403    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02897304-5217-4697-8e96-9d0cf4ec83c1-kube-api-access-8455z" (OuterVolumeSpecName: "kube-api-access-8455z") pod "02897304-5217-4697-8e96-9d0cf4ec83c1" (UID: "02897304-5217-4697-8e96-9d0cf4ec83c1"). InnerVolumeSpecName "kube-api-access-8455z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:42:29.797954 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.797923    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:29.865934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.865905    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8455z\" (UniqueName: \"kubernetes.io/projected/02897304-5217-4697-8e96-9d0cf4ec83c1-kube-api-access-8455z\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:29.866057 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:29.865936    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/02897304-5217-4697-8e96-9d0cf4ec83c1-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:42:30.229986 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.229949    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:30.295097 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.295064    2551 generic.go:356] "Generic (PLEG): container finished" podID="02897304-5217-4697-8e96-9d0cf4ec83c1" containerID="bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9" exitCode=0
Sep 10 13:42:30.295267 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.295114    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"
Sep 10 13:42:30.295267 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.295146    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb" event={"ID":"02897304-5217-4697-8e96-9d0cf4ec83c1","Type":"ContainerDied","Data":"bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9"}
Sep 10 13:42:30.295267 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.295209    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb" event={"ID":"02897304-5217-4697-8e96-9d0cf4ec83c1","Type":"ContainerDied","Data":"8eeee99143b267adaf83d229069a742fb31274b4b6d350bbd4cf413989f2be1a"}
Sep 10 13:42:30.295267 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.295237    2551 scope.go:117] "RemoveContainer" containerID="bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9"
Sep 10 13:42:30.296989 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.296965    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4" event={"ID":"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269","Type":"ContainerStarted","Data":"2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08"}
Sep 10 13:42:30.297065 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.297023    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:30.297065 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.297047    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4" event={"ID":"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269","Type":"ContainerStarted","Data":"8d490cabe10b858c4c7fc181a8fe34f75de3aeffb01763b6cf2cb8793280707b"}
Sep 10 13:42:30.305080 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.305069    2551 scope.go:117] "RemoveContainer" containerID="bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9"
Sep 10 13:42:30.305337 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:42:30.305316    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9\": container with ID starting with bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9 not found: ID does not exist" containerID="bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9"
Sep 10 13:42:30.305407 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.305345    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9"} err="failed to get container status \"bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9\": rpc error: code = NotFound desc = could not find container \"bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9\": container with ID starting with bc0b746bb873e15e04dd2aca415885f722228407629069bb2a63e1d76c88f7a9 not found: ID does not exist"
Sep 10 13:42:30.334575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.334535    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4" podStartSLOduration=2.334522148 podStartE2EDuration="2.334522148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:42:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:42:30.333123321 +0000 UTC m=+1515.544316578" watchObservedRunningTime="2026-09-10 13:42:30.334522148 +0000 UTC m=+1515.545715412"
Sep 10 13:42:30.362215 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.362194    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"]
Sep 10 13:42:30.367793 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.367775    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-e65037-6b75cd48f8-rwsgb"]
Sep 10 13:42:30.798751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:30.798724    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:31.248201 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:31.248114    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02897304-5217-4697-8e96-9d0cf4ec83c1" path="/var/lib/kubelet/pods/02897304-5217-4697-8e96-9d0cf4ec83c1/volumes"
Sep 10 13:42:31.248506 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:31.248485    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55af0bf9-8ace-4bf9-8c42-97c97989bff2" path="/var/lib/kubelet/pods/55af0bf9-8ace-4bf9-8c42-97c97989bff2/volumes"
Sep 10 13:42:37.653263 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:37.653227    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:39.773890 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:39.773854    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:41.307878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:41.307845    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:42:41.392084 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:41.392048    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:42:44.367231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:44.367197    2551 generic.go:356] "Generic (PLEG): container finished" podID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerID="6ff5e650a73d39d6337f29a26fc9c774401cfce8d86ff75dc4d87dcc9b9d0a2f" exitCode=0
Sep 10 13:42:44.367231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:44.367240    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" event={"ID":"4c9f8baf-6a23-46cc-93ce-1c899a25988f","Type":"ContainerDied","Data":"6ff5e650a73d39d6337f29a26fc9c774401cfce8d86ff75dc4d87dcc9b9d0a2f"}
Sep 10 13:42:44.367838 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:44.367739    2551 scope.go:117] "RemoveContainer" containerID="6ff5e650a73d39d6337f29a26fc9c774401cfce8d86ff75dc4d87dcc9b9d0a2f"
Sep 10 13:42:44.369701 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:44.369655    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:44.369776 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:44.369742    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:44.369814 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:44.369775    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 13:42:45.382056 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:45.382023    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" event={"ID":"4c9f8baf-6a23-46cc-93ce-1c899a25988f","Type":"ContainerStarted","Data":"cf13ee6d38c0aca2536f1dc0d11a1bde46714390b66804f2055007c7558270d4"}
Sep 10 13:42:45.408505 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:45.408466    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:45.409521 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:45.409493    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:42:45.409774 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:45.409748    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:42:46.404372 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:46.404339    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:42:46.404767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:46.404386    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:42:47.404595 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:47.404554    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:42:47.405078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:47.404621    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:42:48.404541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:48.404502    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:42:48.404767 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:48.404563    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:42:49.404155 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:49.404117    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Sep 10 13:42:49.404326 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:49.404184    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Sep 10 13:42:50.538558 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:50.538526    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:50.539030 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:50.538940    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:50.539618 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:50.539598    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:42:51.399762 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:51.399732    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:53.090748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:53.090715    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:42:53.197910 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:42:53.197876    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:00.616290 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:00.616257    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:08.341843 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.341814    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"]
Sep 10 13:43:08.342314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.342166    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02897304-5217-4697-8e96-9d0cf4ec83c1" containerName="payload-pre-processing"
Sep 10 13:43:08.342314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.342178    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="02897304-5217-4697-8e96-9d0cf4ec83c1" containerName="payload-pre-processing"
Sep 10 13:43:08.342314 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.342244    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="02897304-5217-4697-8e96-9d0cf4ec83c1" containerName="payload-pre-processing"
Sep 10 13:43:08.348415 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.348396    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.351950 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.351926    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-56b405\"/\"default-dockercfg-bl6xl\""
Sep 10 13:43:08.351950 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.351945    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-56b405\"/\"kube-root-ca.crt\""
Sep 10 13:43:08.353390 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.353375    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-56b405\"/\"openshift-service-ca.crt\""
Sep 10 13:43:08.353584 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.353565    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-56b405\"/\"ipp-route-56b405-kserve-self-signed-certs\""
Sep 10 13:43:08.358150 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.358130    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"]
Sep 10 13:43:08.395872 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.395843    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-model-cache\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.396005 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.395889    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-home\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.396061 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.396001    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-tmp-dir\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.396061 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.396033    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-dshm\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.396061 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.396056    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c778419-b1b0-4eb9-ab1d-4330899a4658-tls-certs\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.396177 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.396085    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwvl9\" (UniqueName: \"kubernetes.io/projected/8c778419-b1b0-4eb9-ab1d-4330899a4658-kube-api-access-dwvl9\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.453460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.453440    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:08.496397 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496374    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-home\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.496495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496415    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-tmp-dir\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.496495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496436    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-dshm\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.496495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496467    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c778419-b1b0-4eb9-ab1d-4330899a4658-tls-certs\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.496648 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496507    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwvl9\" (UniqueName: \"kubernetes.io/projected/8c778419-b1b0-4eb9-ab1d-4330899a4658-kube-api-access-dwvl9\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.496648 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496564    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-model-cache\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.496828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496807    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-home\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.496894 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496848    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-tmp-dir\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.497016 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.496997    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-model-cache\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.499256 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.499236    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-dshm\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.499440 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.499423    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c778419-b1b0-4eb9-ab1d-4330899a4658-tls-certs\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.512304 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.512285    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwvl9\" (UniqueName: \"kubernetes.io/projected/8c778419-b1b0-4eb9-ab1d-4330899a4658-kube-api-access-dwvl9\") pod \"ipp-route-56b405-kserve-58f986b8f9-rjsh9\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") " pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.658247 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.658176    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:08.937724 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:08.937642    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"]
Sep 10 13:43:08.940703 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:43:08.940675    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c778419_b1b0_4eb9_ab1d_4330899a4658.slice/crio-77e19c7d88b8cb654475c712021146842c7247f509efa46727f042e5ec256f48 WatchSource:0}: Error finding container 77e19c7d88b8cb654475c712021146842c7247f509efa46727f042e5ec256f48: Status 404 returned error can't find the container with id 77e19c7d88b8cb654475c712021146842c7247f509efa46727f042e5ec256f48
Sep 10 13:43:09.503878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:09.503847    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9" event={"ID":"8c778419-b1b0-4eb9-ab1d-4330899a4658","Type":"ContainerStarted","Data":"7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1"}
Sep 10 13:43:09.503878 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:09.503884    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9" event={"ID":"8c778419-b1b0-4eb9-ab1d-4330899a4658","Type":"ContainerStarted","Data":"77e19c7d88b8cb654475c712021146842c7247f509efa46727f042e5ec256f48"}
Sep 10 13:43:09.543308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:09.543259    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9" podStartSLOduration=1.543243793 podStartE2EDuration="1.543243793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:09.543014306 +0000 UTC m=+1554.754207563" watchObservedRunningTime="2026-09-10 13:43:09.543243793 +0000 UTC m=+1554.754437089"
Sep 10 13:43:12.471340 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:12.471308    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:12.935240 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:12.935209    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:43:18.659299 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:18.659266    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:18.659783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:18.659312    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:18.671588 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:18.671568    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:19.559337 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:19.559306    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:43:26.213588 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:26.213561    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:43:32.593935 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:32.593903    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v_d192a663-422e-4a90-a620-7bfe159a9117/payload-processing/0.log"
Sep 10 13:43:33.355897 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:33.355865    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:33.597346 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:33.597315    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:33.936850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:33.936820    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:36.588150 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:36.588117    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:36.672019 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:36.671994    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:36.738469 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:36.738442    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:37.785728 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:37.785691    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:37.996863 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:37.996834    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:38.101496 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.101419    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:38.322351 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.322323    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:38.415542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.415440    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:43:38.497797 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.497768    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"]
Sep 10 13:43:38.501535 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.501514    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:38.508578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.505863    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-37280b\""
Sep 10 13:43:38.508578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.507692    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-37280b-dockercfg-wn949\""
Sep 10 13:43:38.519793 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.519769    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"]
Sep 10 13:43:38.635536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.635497    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd227ec-ca2c-42f8-917f-8b52e58640d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:38.635721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.635545    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clnng\" (UniqueName: \"kubernetes.io/projected/ffd227ec-ca2c-42f8-917f-8b52e58640d7-kube-api-access-clnng\") pod \"payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:38.736783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.736720    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd227ec-ca2c-42f8-917f-8b52e58640d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:38.736783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.736759    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clnng\" (UniqueName: \"kubernetes.io/projected/ffd227ec-ca2c-42f8-917f-8b52e58640d7-kube-api-access-clnng\") pod \"payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:38.737384 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.737362    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd227ec-ca2c-42f8-917f-8b52e58640d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:38.749532 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.749513    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clnng\" (UniqueName: \"kubernetes.io/projected/ffd227ec-ca2c-42f8-917f-8b52e58640d7-kube-api-access-clnng\") pod \"payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:38.818558 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:38.818530    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:39.094673 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:39.094630    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"]
Sep 10 13:43:39.096517 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:43:39.096487    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd227ec_ca2c_42f8_917f_8b52e58640d7.slice/crio-26b731dec979a3a1837577820512814090a473f677000c790a8c87f61ea34d5e WatchSource:0}: Error finding container 26b731dec979a3a1837577820512814090a473f677000c790a8c87f61ea34d5e: Status 404 returned error can't find the container with id 26b731dec979a3a1837577820512814090a473f677000c790a8c87f61ea34d5e
Sep 10 13:43:39.640875 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:39.640844    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb" event={"ID":"ffd227ec-ca2c-42f8-917f-8b52e58640d7","Type":"ContainerStarted","Data":"9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d"}
Sep 10 13:43:39.641108 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:39.640892    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb" event={"ID":"ffd227ec-ca2c-42f8-917f-8b52e58640d7","Type":"ContainerStarted","Data":"26b731dec979a3a1837577820512814090a473f677000c790a8c87f61ea34d5e"}
Sep 10 13:43:39.641108 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:39.640996    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:43:39.661289 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:39.661244    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb" podStartSLOduration=1.661225097 podStartE2EDuration="1.661225097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:43:39.660141125 +0000 UTC m=+1584.871334380" watchObservedRunningTime="2026-09-10 13:43:39.661225097 +0000 UTC m=+1584.872418353"
Sep 10 13:43:45.613447 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:45.613412    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:43:50.648699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:43:50.648647    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:44:01.636833 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:01.636800    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:08.658611 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:08.658518    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:08.713750 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:08.713716    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"]
Sep 10 13:44:08.713975 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:08.713950    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb" podUID="ffd227ec-ca2c-42f8-917f-8b52e58640d7" containerName="payload-pre-processing" containerID="cri-o://9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d" gracePeriod=30
Sep 10 13:44:08.739288 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:08.739263    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:08.921636 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:08.921555    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:08.957879 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:08.957856    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:44:09.010061 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.010026    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clnng\" (UniqueName: \"kubernetes.io/projected/ffd227ec-ca2c-42f8-917f-8b52e58640d7-kube-api-access-clnng\") pod \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") "
Sep 10 13:44:09.010212 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.010123    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd227ec-ca2c-42f8-917f-8b52e58640d7-plugins-config-volume\") pod \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\" (UID: \"ffd227ec-ca2c-42f8-917f-8b52e58640d7\") "
Sep 10 13:44:09.010571 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.010539    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffd227ec-ca2c-42f8-917f-8b52e58640d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ffd227ec-ca2c-42f8-917f-8b52e58640d7" (UID: "ffd227ec-ca2c-42f8-917f-8b52e58640d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:44:09.012021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.011992    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffd227ec-ca2c-42f8-917f-8b52e58640d7-kube-api-access-clnng" (OuterVolumeSpecName: "kube-api-access-clnng") pod "ffd227ec-ca2c-42f8-917f-8b52e58640d7" (UID: "ffd227ec-ca2c-42f8-917f-8b52e58640d7"). InnerVolumeSpecName "kube-api-access-clnng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:09.054262 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.053947    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:09.110792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.110756    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clnng\" (UniqueName: \"kubernetes.io/projected/ffd227ec-ca2c-42f8-917f-8b52e58640d7-kube-api-access-clnng\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:09.110792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.110791    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ffd227ec-ca2c-42f8-917f-8b52e58640d7-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:09.483010 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.482975    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:09.789469 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.789436    2551 generic.go:356] "Generic (PLEG): container finished" podID="ffd227ec-ca2c-42f8-917f-8b52e58640d7" containerID="9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d" exitCode=0
Sep 10 13:44:09.789923 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.789481    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"
Sep 10 13:44:09.789923 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.789516    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb" event={"ID":"ffd227ec-ca2c-42f8-917f-8b52e58640d7","Type":"ContainerDied","Data":"9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d"}
Sep 10 13:44:09.789923 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.789569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb" event={"ID":"ffd227ec-ca2c-42f8-917f-8b52e58640d7","Type":"ContainerDied","Data":"26b731dec979a3a1837577820512814090a473f677000c790a8c87f61ea34d5e"}
Sep 10 13:44:09.789923 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.789592    2551 scope.go:117] "RemoveContainer" containerID="9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d"
Sep 10 13:44:09.798341 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.798324    2551 scope.go:117] "RemoveContainer" containerID="9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d"
Sep 10 13:44:09.798630 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:44:09.798611    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d\": container with ID starting with 9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d not found: ID does not exist" containerID="9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d"
Sep 10 13:44:09.798709 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.798638    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d"} err="failed to get container status \"9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d\": rpc error: code = NotFound desc = could not find container \"9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d\": container with ID starting with 9ef081ebdf2268892d779b26469f102ef6e37de6700fc63e74379d696d4f356d not found: ID does not exist"
Sep 10 13:44:09.809893 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.809870    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"]
Sep 10 13:44:09.812820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:09.812799    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-37280b-5d6f68fc94-xhnwb"]
Sep 10 13:44:11.249040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:11.248997    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffd227ec-ca2c-42f8-917f-8b52e58640d7" path="/var/lib/kubelet/pods/ffd227ec-ca2c-42f8-917f-8b52e58640d7/volumes"
Sep 10 13:44:13.406539 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:13.406509    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:44:15.516245 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:15.516213    2551 scope.go:117] "RemoveContainer" containerID="b273cfb9e7aaafdd6fffa9fd915a7bb05d774fa58bd5bfa5c517924bb768549f"
Sep 10 13:44:16.096490 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:16.096455    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:16.197986 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:16.197934    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:16.388565 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:16.388462    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:20.645081 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:20.645046    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:33.900233 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:33.900190    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:44:36.117014 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.116979    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:36.159828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.159791    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"]
Sep 10 13:44:36.160066 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.160039    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v" podUID="d192a663-422e-4a90-a620-7bfe159a9117" containerName="payload-processing" containerID="cri-o://883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8" gracePeriod=30
Sep 10 13:44:36.164831 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.164717    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"]
Sep 10 13:44:36.165078 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.164955    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp" podUID="84a875d7-c911-4e32-9d3f-c308859cdd34" containerName="payload-pre-processing" containerID="cri-o://ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78" gracePeriod=30
Sep 10 13:44:36.204728 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.204696    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:36.269292 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.269248    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:36.364286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.364249    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:36.425200 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.425171    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:36.431525 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.431503    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:44:36.452195 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.452167    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:44:36.561084 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.561056    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqg68\" (UniqueName: \"kubernetes.io/projected/d192a663-422e-4a90-a620-7bfe159a9117-kube-api-access-jqg68\") pod \"d192a663-422e-4a90-a620-7bfe159a9117\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") "
Sep 10 13:44:36.561269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.561108    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pb2n2\" (UniqueName: \"kubernetes.io/projected/84a875d7-c911-4e32-9d3f-c308859cdd34-kube-api-access-pb2n2\") pod \"84a875d7-c911-4e32-9d3f-c308859cdd34\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") "
Sep 10 13:44:36.561269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.561145    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d192a663-422e-4a90-a620-7bfe159a9117-plugins-config-volume\") pod \"d192a663-422e-4a90-a620-7bfe159a9117\" (UID: \"d192a663-422e-4a90-a620-7bfe159a9117\") "
Sep 10 13:44:36.561269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.561220    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a875d7-c911-4e32-9d3f-c308859cdd34-plugins-config-volume\") pod \"84a875d7-c911-4e32-9d3f-c308859cdd34\" (UID: \"84a875d7-c911-4e32-9d3f-c308859cdd34\") "
Sep 10 13:44:36.561582 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.561560    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d192a663-422e-4a90-a620-7bfe159a9117-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d192a663-422e-4a90-a620-7bfe159a9117" (UID: "d192a663-422e-4a90-a620-7bfe159a9117"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:44:36.561697 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.561610    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84a875d7-c911-4e32-9d3f-c308859cdd34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84a875d7-c911-4e32-9d3f-c308859cdd34" (UID: "84a875d7-c911-4e32-9d3f-c308859cdd34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:44:36.563186 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.563166    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d192a663-422e-4a90-a620-7bfe159a9117-kube-api-access-jqg68" (OuterVolumeSpecName: "kube-api-access-jqg68") pod "d192a663-422e-4a90-a620-7bfe159a9117" (UID: "d192a663-422e-4a90-a620-7bfe159a9117"). InnerVolumeSpecName "kube-api-access-jqg68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:36.563186 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.563182    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a875d7-c911-4e32-9d3f-c308859cdd34-kube-api-access-pb2n2" (OuterVolumeSpecName: "kube-api-access-pb2n2") pod "84a875d7-c911-4e32-9d3f-c308859cdd34" (UID: "84a875d7-c911-4e32-9d3f-c308859cdd34"). InnerVolumeSpecName "kube-api-access-pb2n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:36.662499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.662437    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d192a663-422e-4a90-a620-7bfe159a9117-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:36.662499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.662460    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84a875d7-c911-4e32-9d3f-c308859cdd34-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:36.662499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.662469    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqg68\" (UniqueName: \"kubernetes.io/projected/d192a663-422e-4a90-a620-7bfe159a9117-kube-api-access-jqg68\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:36.662499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.662477    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pb2n2\" (UniqueName: \"kubernetes.io/projected/84a875d7-c911-4e32-9d3f-c308859cdd34-kube-api-access-pb2n2\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:36.763978 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.763949    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:36.920783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.920709    2551 generic.go:356] "Generic (PLEG): container finished" podID="84a875d7-c911-4e32-9d3f-c308859cdd34" containerID="ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78" exitCode=0
Sep 10 13:44:36.920783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.920758    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"
Sep 10 13:44:36.920973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.920808    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp" event={"ID":"84a875d7-c911-4e32-9d3f-c308859cdd34","Type":"ContainerDied","Data":"ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78"}
Sep 10 13:44:36.920973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.920860    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp" event={"ID":"84a875d7-c911-4e32-9d3f-c308859cdd34","Type":"ContainerDied","Data":"38864bad59d94a05f36c1f26c62f1c998122f59c3f0bd9fb3f723444a3efd649"}
Sep 10 13:44:36.920973 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.920885    2551 scope.go:117] "RemoveContainer" containerID="ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78"
Sep 10 13:44:36.922251 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.922227    2551 generic.go:356] "Generic (PLEG): container finished" podID="d192a663-422e-4a90-a620-7bfe159a9117" containerID="883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8" exitCode=0
Sep 10 13:44:36.922376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.922273    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"
Sep 10 13:44:36.922376 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.922325    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v" event={"ID":"d192a663-422e-4a90-a620-7bfe159a9117","Type":"ContainerDied","Data":"883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8"}
Sep 10 13:44:36.922468 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.922378    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v" event={"ID":"d192a663-422e-4a90-a620-7bfe159a9117","Type":"ContainerDied","Data":"9cc3aec87a6b10c216d2ea144521fd4e0cf310c3f524d46b09160683007c32a3"}
Sep 10 13:44:36.929692 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.929585    2551 scope.go:117] "RemoveContainer" containerID="ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78"
Sep 10 13:44:36.929937 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:44:36.929920    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78\": container with ID starting with ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78 not found: ID does not exist" containerID="ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78"
Sep 10 13:44:36.929995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.929946    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78"} err="failed to get container status \"ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78\": rpc error: code = NotFound desc = could not find container \"ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78\": container with ID starting with ea60e0b67ed9eefd1239eac37630cea9b0a19d74549a1f49666fde29f15cbc78 not found: ID does not exist"
Sep 10 13:44:36.929995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.929968    2551 scope.go:117] "RemoveContainer" containerID="883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8"
Sep 10 13:44:36.937529 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.937514    2551 scope.go:117] "RemoveContainer" containerID="883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8"
Sep 10 13:44:36.937776 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:44:36.937760    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8\": container with ID starting with 883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8 not found: ID does not exist" containerID="883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8"
Sep 10 13:44:36.937820 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.937782    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8"} err="failed to get container status \"883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8\": rpc error: code = NotFound desc = could not find container \"883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8\": container with ID starting with 883e595d9ff157c197b139147744edde3d2d0fb5b7fdc7669a650e00d4e2ccf8 not found: ID does not exist"
Sep 10 13:44:36.945954 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.945934    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"]
Sep 10 13:44:36.949711 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.949690    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-56b405-5cd7b458b-snllp"]
Sep 10 13:44:36.960884 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.960864    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"]
Sep 10 13:44:36.967243 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:36.967225    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-56b405-c4bbfdfb9-4z92v"]
Sep 10 13:44:37.248326 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:37.248299    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a875d7-c911-4e32-9d3f-c308859cdd34" path="/var/lib/kubelet/pods/84a875d7-c911-4e32-9d3f-c308859cdd34/volumes"
Sep 10 13:44:37.248650 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:37.248637    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d192a663-422e-4a90-a620-7bfe159a9117" path="/var/lib/kubelet/pods/d192a663-422e-4a90-a620-7bfe159a9117/volumes"
Sep 10 13:44:41.746640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:41.746602    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"]
Sep 10 13:44:41.843546 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:41.843495    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:44:47.606272 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:47.606232    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:44:56.766087 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:56.766026    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9" podUID="8c778419-b1b0-4eb9-ab1d-4330899a4658" containerName="main" containerID="cri-o://7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1" gracePeriod=45
Sep 10 13:44:57.008703 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.008657    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:44:57.016044 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.016017    2551 generic.go:356] "Generic (PLEG): container finished" podID="8c778419-b1b0-4eb9-ab1d-4330899a4658" containerID="7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1" exitCode=0
Sep 10 13:44:57.016210 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.016052    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9" event={"ID":"8c778419-b1b0-4eb9-ab1d-4330899a4658","Type":"ContainerDied","Data":"7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1"}
Sep 10 13:44:57.016210 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.016059    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"
Sep 10 13:44:57.016210 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.016078    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9" event={"ID":"8c778419-b1b0-4eb9-ab1d-4330899a4658","Type":"ContainerDied","Data":"77e19c7d88b8cb654475c712021146842c7247f509efa46727f042e5ec256f48"}
Sep 10 13:44:57.016210 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.016094    2551 scope.go:117] "RemoveContainer" containerID="7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1"
Sep 10 13:44:57.019269 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019247    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-home\") pod \"8c778419-b1b0-4eb9-ab1d-4330899a4658\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") "
Sep 10 13:44:57.019382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019273    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-dshm\") pod \"8c778419-b1b0-4eb9-ab1d-4330899a4658\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") "
Sep 10 13:44:57.019382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019320    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-tmp-dir\") pod \"8c778419-b1b0-4eb9-ab1d-4330899a4658\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") "
Sep 10 13:44:57.019382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019347    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-model-cache\") pod \"8c778419-b1b0-4eb9-ab1d-4330899a4658\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") "
Sep 10 13:44:57.019382 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019376    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c778419-b1b0-4eb9-ab1d-4330899a4658-tls-certs\") pod \"8c778419-b1b0-4eb9-ab1d-4330899a4658\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") "
Sep 10 13:44:57.019591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019435    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwvl9\" (UniqueName: \"kubernetes.io/projected/8c778419-b1b0-4eb9-ab1d-4330899a4658-kube-api-access-dwvl9\") pod \"8c778419-b1b0-4eb9-ab1d-4330899a4658\" (UID: \"8c778419-b1b0-4eb9-ab1d-4330899a4658\") "
Sep 10 13:44:57.019591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019489    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-home" (OuterVolumeSpecName: "home") pod "8c778419-b1b0-4eb9-ab1d-4330899a4658" (UID: "8c778419-b1b0-4eb9-ab1d-4330899a4658"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:44:57.019730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019610    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-model-cache" (OuterVolumeSpecName: "model-cache") pod "8c778419-b1b0-4eb9-ab1d-4330899a4658" (UID: "8c778419-b1b0-4eb9-ab1d-4330899a4658"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:44:57.019730 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019619    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8c778419-b1b0-4eb9-ab1d-4330899a4658" (UID: "8c778419-b1b0-4eb9-ab1d-4330899a4658"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:44:57.019828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019768    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:57.019828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019790    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:57.019828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.019807    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:57.021637 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.021614    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c778419-b1b0-4eb9-ab1d-4330899a4658-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8c778419-b1b0-4eb9-ab1d-4330899a4658" (UID: "8c778419-b1b0-4eb9-ab1d-4330899a4658"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:44:57.021990 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.021971    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-dshm" (OuterVolumeSpecName: "dshm") pod "8c778419-b1b0-4eb9-ab1d-4330899a4658" (UID: "8c778419-b1b0-4eb9-ab1d-4330899a4658"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:44:57.022060 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.021998    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c778419-b1b0-4eb9-ab1d-4330899a4658-kube-api-access-dwvl9" (OuterVolumeSpecName: "kube-api-access-dwvl9") pod "8c778419-b1b0-4eb9-ab1d-4330899a4658" (UID: "8c778419-b1b0-4eb9-ab1d-4330899a4658"). InnerVolumeSpecName "kube-api-access-dwvl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:44:57.025510 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.025491    2551 scope.go:117] "RemoveContainer" containerID="7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1"
Sep 10 13:44:57.025826 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:44:57.025807    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1\": container with ID starting with 7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1 not found: ID does not exist" containerID="7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1"
Sep 10 13:44:57.025898 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.025838    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1"} err="failed to get container status \"7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1\": rpc error: code = NotFound desc = could not find container \"7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1\": container with ID starting with 7ba84f9da54a9dd51ca50dd97fe8d0a971e3dba092b548644aff0cac39a2a6d1 not found: ID does not exist"
Sep 10 13:44:57.120842 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.120813    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8c778419-b1b0-4eb9-ab1d-4330899a4658-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:57.120842 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.120836    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwvl9\" (UniqueName: \"kubernetes.io/projected/8c778419-b1b0-4eb9-ab1d-4330899a4658-kube-api-access-dwvl9\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:57.120842 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.120846    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8c778419-b1b0-4eb9-ab1d-4330899a4658-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:44:57.333241 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.333211    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"]
Sep 10 13:44:57.339731 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:57.339710    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-56b405/ipp-route-56b405-kserve-58f986b8f9-rjsh9"]
Sep 10 13:44:59.248475 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:44:59.248443    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c778419-b1b0-4eb9-ab1d-4330899a4658" path="/var/lib/kubelet/pods/8c778419-b1b0-4eb9-ab1d-4330899a4658/volumes"
Sep 10 13:45:00.154434 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154400    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"]
Sep 10 13:45:00.154771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154755    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a875d7-c911-4e32-9d3f-c308859cdd34" containerName="payload-pre-processing"
Sep 10 13:45:00.154832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154772    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a875d7-c911-4e32-9d3f-c308859cdd34" containerName="payload-pre-processing"
Sep 10 13:45:00.154832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154784    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d192a663-422e-4a90-a620-7bfe159a9117" containerName="payload-processing"
Sep 10 13:45:00.154832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154790    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d192a663-422e-4a90-a620-7bfe159a9117" containerName="payload-processing"
Sep 10 13:45:00.154832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154801    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c778419-b1b0-4eb9-ab1d-4330899a4658" containerName="main"
Sep 10 13:45:00.154832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154807    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c778419-b1b0-4eb9-ab1d-4330899a4658" containerName="main"
Sep 10 13:45:00.154832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154820    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffd227ec-ca2c-42f8-917f-8b52e58640d7" containerName="payload-pre-processing"
Sep 10 13:45:00.154832 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154826    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffd227ec-ca2c-42f8-917f-8b52e58640d7" containerName="payload-pre-processing"
Sep 10 13:45:00.155042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154880    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a875d7-c911-4e32-9d3f-c308859cdd34" containerName="payload-pre-processing"
Sep 10 13:45:00.155042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154888    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffd227ec-ca2c-42f8-917f-8b52e58640d7" containerName="payload-pre-processing"
Sep 10 13:45:00.155042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154893    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c778419-b1b0-4eb9-ab1d-4330899a4658" containerName="main"
Sep 10 13:45:00.155042 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.154901    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d192a663-422e-4a90-a620-7bfe159a9117" containerName="payload-processing"
Sep 10 13:45:00.161561 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.161528    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"]
Sep 10 13:45:00.161834 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.161811    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"
Sep 10 13:45:00.165049 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.165026    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"
Sep 10 13:45:00.165325 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.165301    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:45:00.170105 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.170082    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"]
Sep 10 13:45:00.172357 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.172334    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"]
Sep 10 13:45:00.175279 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.175262    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"
Sep 10 13:45:00.182253 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.182233    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"
Sep 10 13:45:00.461783 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.461759    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"]
Sep 10 13:45:00.464309 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:45:00.464275    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfb5e47_4be3_405c_898e_a130cd5feeb2.slice/crio-cc31ef1cbbf45eeb23eca99fe795cce303e5c7f96fbc0b5fe2837a48aead6956 WatchSource:0}: Error finding container cc31ef1cbbf45eeb23eca99fe795cce303e5c7f96fbc0b5fe2837a48aead6956: Status 404 returned error can't find the container with id cc31ef1cbbf45eeb23eca99fe795cce303e5c7f96fbc0b5fe2837a48aead6956
Sep 10 13:45:00.465175 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:00.465152    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"]
Sep 10 13:45:00.466766 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:45:00.466740    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ecdd9e_5f8b_498a_8a72_ed92da5d9338.slice/crio-c13a669cefca16cae33b8054e88f65da6fbfc747e7869a027c2db9045d6f43dd WatchSource:0}: Error finding container c13a669cefca16cae33b8054e88f65da6fbfc747e7869a027c2db9045d6f43dd: Status 404 returned error can't find the container with id c13a669cefca16cae33b8054e88f65da6fbfc747e7869a027c2db9045d6f43dd
Sep 10 13:45:01.038222 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:01.038189    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m" event={"ID":"ecfb5e47-4be3-405c-898e-a130cd5feeb2","Type":"ContainerStarted","Data":"fff0e84e8a12be832132b0ab32b19d614dbcb90e7ed64d681311e8b63f22ff8e"}
Sep 10 13:45:01.038415 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:01.038239    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m" event={"ID":"ecfb5e47-4be3-405c-898e-a130cd5feeb2","Type":"ContainerStarted","Data":"cc31ef1cbbf45eeb23eca99fe795cce303e5c7f96fbc0b5fe2837a48aead6956"}
Sep 10 13:45:01.039711 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:01.039693    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq" event={"ID":"c6ecdd9e-5f8b-498a-8a72-ed92da5d9338","Type":"ContainerStarted","Data":"191c6635cf3abf3ea1c285d5c1bbd56c907848c03620e465050325e897f7637c"}
Sep 10 13:45:01.039821 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:01.039726    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq" event={"ID":"c6ecdd9e-5f8b-498a-8a72-ed92da5d9338","Type":"ContainerStarted","Data":"c13a669cefca16cae33b8054e88f65da6fbfc747e7869a027c2db9045d6f43dd"}
Sep 10 13:45:01.060732 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:01.060681    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m" podStartSLOduration=1.060651157 podStartE2EDuration="1.060651157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:01.058704762 +0000 UTC m=+1666.269898020" watchObservedRunningTime="2026-09-10 13:45:01.060651157 +0000 UTC m=+1666.271844449"
Sep 10 13:45:01.074683 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:01.074620    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq" podStartSLOduration=1.074605109 podStartE2EDuration="1.074605109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:45:01.073742339 +0000 UTC m=+1666.284935595" watchObservedRunningTime="2026-09-10 13:45:01.074605109 +0000 UTC m=+1666.285798405"
Sep 10 13:45:02.045715 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:02.045652    2551 generic.go:356] "Generic (PLEG): container finished" podID="ecfb5e47-4be3-405c-898e-a130cd5feeb2" containerID="fff0e84e8a12be832132b0ab32b19d614dbcb90e7ed64d681311e8b63f22ff8e" exitCode=0
Sep 10 13:45:02.046248 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:02.045733    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m" event={"ID":"ecfb5e47-4be3-405c-898e-a130cd5feeb2","Type":"ContainerDied","Data":"fff0e84e8a12be832132b0ab32b19d614dbcb90e7ed64d681311e8b63f22ff8e"}
Sep 10 13:45:02.047274 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:02.047251    2551 generic.go:356] "Generic (PLEG): container finished" podID="c6ecdd9e-5f8b-498a-8a72-ed92da5d9338" containerID="191c6635cf3abf3ea1c285d5c1bbd56c907848c03620e465050325e897f7637c" exitCode=0
Sep 10 13:45:02.047414 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:02.047301    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq" event={"ID":"c6ecdd9e-5f8b-498a-8a72-ed92da5d9338","Type":"ContainerDied","Data":"191c6635cf3abf3ea1c285d5c1bbd56c907848c03620e465050325e897f7637c"}
Sep 10 13:45:03.204752 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:03.204730    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"
Sep 10 13:45:03.210092 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:03.210074    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"
Sep 10 13:45:04.057547 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:04.057507    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m" event={"ID":"ecfb5e47-4be3-405c-898e-a130cd5feeb2","Type":"ContainerDied","Data":"cc31ef1cbbf45eeb23eca99fe795cce303e5c7f96fbc0b5fe2837a48aead6956"}
Sep 10 13:45:04.057771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:04.057569    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc31ef1cbbf45eeb23eca99fe795cce303e5c7f96fbc0b5fe2837a48aead6956"
Sep 10 13:45:04.057771 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:04.057516    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"
Sep 10 13:45:04.058995 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:04.058974    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"
Sep 10 13:45:04.059105 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:04.058970    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq" event={"ID":"c6ecdd9e-5f8b-498a-8a72-ed92da5d9338","Type":"ContainerDied","Data":"c13a669cefca16cae33b8054e88f65da6fbfc747e7869a027c2db9045d6f43dd"}
Sep 10 13:45:04.059105 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:04.059098    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c13a669cefca16cae33b8054e88f65da6fbfc747e7869a027c2db9045d6f43dd"
Sep 10 13:45:07.220998 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:07.220966    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:18.685483 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:18.685449    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:19.736791 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:19.736761    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:26.041418 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:26.041366    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:26.113514 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:26.113484    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:26.411064 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:26.410943    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:27.020994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:27.020958    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:37.513333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.513297    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:37.552868 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.552836    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"]
Sep 10 13:45:37.553103 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.553079    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4" podUID="ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" containerName="payload-pre-processing" containerID="cri-o://2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08" gracePeriod=30
Sep 10 13:45:37.615841 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.615810    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:37.716369 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.716339    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:37.802348 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.802295    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:45:37.840855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.840822    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:37.940133 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.940100    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4vsz\" (UniqueName: \"kubernetes.io/projected/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-kube-api-access-k4vsz\") pod \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") "
Sep 10 13:45:37.940133 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.940136    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-plugins-config-volume\") pod \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\" (UID: \"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269\") "
Sep 10 13:45:37.940571 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.940542    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" (UID: "ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:45:37.942338 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:37.942311    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-kube-api-access-k4vsz" (OuterVolumeSpecName: "kube-api-access-k4vsz") pod "ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" (UID: "ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269"). InnerVolumeSpecName "kube-api-access-k4vsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:38.041164 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.041091    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4vsz\" (UniqueName: \"kubernetes.io/projected/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-kube-api-access-k4vsz\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:38.041164 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.041118    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:38.166075 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.166045    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:38.224172 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.224142    2551 generic.go:356] "Generic (PLEG): container finished" podID="ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" containerID="2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08" exitCode=0
Sep 10 13:45:38.224333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.224196    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"
Sep 10 13:45:38.224333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.224227    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4" event={"ID":"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269","Type":"ContainerDied","Data":"2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08"}
Sep 10 13:45:38.224333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.224302    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4" event={"ID":"ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269","Type":"ContainerDied","Data":"8d490cabe10b858c4c7fc181a8fe34f75de3aeffb01763b6cf2cb8793280707b"}
Sep 10 13:45:38.224333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.224325    2551 scope.go:117] "RemoveContainer" containerID="2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08"
Sep 10 13:45:38.233322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.233305    2551 scope.go:117] "RemoveContainer" containerID="2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08"
Sep 10 13:45:38.233585 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:45:38.233568    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08\": container with ID starting with 2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08 not found: ID does not exist" containerID="2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08"
Sep 10 13:45:38.233652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.233596    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08"} err="failed to get container status \"2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08\": rpc error: code = NotFound desc = could not find container \"2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08\": container with ID starting with 2940da6787c15ab38970a510cde01a99d945baadb4427b484040577552dc6d08 not found: ID does not exist"
Sep 10 13:45:38.261919 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.261895    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"]
Sep 10 13:45:38.265605 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.265564    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:38.270138 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:38.270119    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-08b85c-7456cf9894-td6d4"]
Sep 10 13:45:39.248157 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:39.248119    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" path="/var/lib/kubelet/pods/ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269/volumes"
Sep 10 13:45:40.991350 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:40.991313    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:45:44.818610 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:44.818541    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:44.914716 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:44.914651    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:45.146569 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:45.146498    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"]
Sep 10 13:45:45.146753 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:45.146731    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy" containerID="cri-o://cf13ee6d38c0aca2536f1dc0d11a1bde46714390b66804f2055007c7558270d4" gracePeriod=30
Sep 10 13:45:45.152425 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:45.152404    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:45.624440 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:45.624401    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:50.009976 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.009947    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:50.282160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.282134    2551 generic.go:356] "Generic (PLEG): container finished" podID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerID="cf13ee6d38c0aca2536f1dc0d11a1bde46714390b66804f2055007c7558270d4" exitCode=0
Sep 10 13:45:50.282290 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.282214    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" event={"ID":"4c9f8baf-6a23-46cc-93ce-1c899a25988f","Type":"ContainerDied","Data":"cf13ee6d38c0aca2536f1dc0d11a1bde46714390b66804f2055007c7558270d4"}
Sep 10 13:45:50.282343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.282305    2551 scope.go:117] "RemoveContainer" containerID="6ff5e650a73d39d6337f29a26fc9c774401cfce8d86ff75dc4d87dcc9b9d0a2f"
Sep 10 13:45:50.405142 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.405120    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:45:50.443913 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.443883    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-podinfo\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.443923    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-data\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.443956    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-envoy\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444021 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.443996    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-credential-socket\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444044    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54w29\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-kube-api-access-54w29\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444087    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-certs\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444169 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444156    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istiod-ca-cert\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444331 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444176    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:50.444331 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444195    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-socket\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444331 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444230    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-token\") pod \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\" (UID: \"4c9f8baf-6a23-46cc-93ce-1c899a25988f\") "
Sep 10 13:45:50.444496 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444452    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:50.444597 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444538    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:45:50.444597 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444572    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-socket\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.444901 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444612    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-data\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.444901 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444633    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:50.444901 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.444755    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:50.446300 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.446266    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:45:50.446447 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.446427    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:50.446575 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.446560    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 13:45:50.446653 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.446631    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-kube-api-access-54w29" (OuterVolumeSpecName: "kube-api-access-54w29") pod "4c9f8baf-6a23-46cc-93ce-1c899a25988f" (UID: "4c9f8baf-6a23-46cc-93ce-1c899a25988f"). InnerVolumeSpecName "kube-api-access-54w29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:50.545240 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.545211    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istiod-ca-cert\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.545240 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.545238    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-token\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.545375 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.545246    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-podinfo\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.545375 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.545254    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-istio-envoy\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.545375 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.545262    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-credential-socket\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.545375 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.545270    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54w29\" (UniqueName: \"kubernetes.io/projected/4c9f8baf-6a23-46cc-93ce-1c899a25988f-kube-api-access-54w29\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:50.545375 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:50.545278    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c9f8baf-6a23-46cc-93ce-1c899a25988f-workload-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:51.287960 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:51.287936    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"
Sep 10 13:45:51.288283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:51.287964    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl" event={"ID":"4c9f8baf-6a23-46cc-93ce-1c899a25988f","Type":"ContainerDied","Data":"7835889a8f0a23a60bd1bc50482d9cddcf8d29f94c58d29566902d533043f728"}
Sep 10 13:45:51.288283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:51.288017    2551 scope.go:117] "RemoveContainer" containerID="cf13ee6d38c0aca2536f1dc0d11a1bde46714390b66804f2055007c7558270d4"
Sep 10 13:45:51.308581 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:51.308557    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"]
Sep 10 13:45:51.313317 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:51.313298    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-08b85c-openshift-default-5f6849fd8d-d2nbl"]
Sep 10 13:45:53.247750 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:53.247715    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" path="/var/lib/kubelet/pods/4c9f8baf-6a23-46cc-93ce-1c899a25988f/volumes"
Sep 10 13:45:56.499896 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.499865    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:56.603052 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.603022    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"]
Sep 10 13:45:56.616333 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.616307    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7657ef-29817465-296bq"]
Sep 10 13:45:56.651931 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.651901    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"]
Sep 10 13:45:56.652146 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.652120    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv" podUID="cfcfd673-94f5-4601-bb1e-e562075c207f" containerName="payload-processing" containerID="cri-o://7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6" gracePeriod=30
Sep 10 13:45:56.667740 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.667716    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"]
Sep 10 13:45:56.667922 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.667879    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d" podUID="65f92dec-a0b2-467b-9b90-c9b218ff556d" containerName="payload-pre-processing" containerID="cri-o://41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc" gracePeriod=30
Sep 10 13:45:56.709488 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.709462    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:56.811278 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.810677    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:56.940731 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.940412    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:45:56.942861 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.942836    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:56.943855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.943837    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:45:56.998035 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.998010    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp9gd\" (UniqueName: \"kubernetes.io/projected/65f92dec-a0b2-467b-9b90-c9b218ff556d-kube-api-access-wp9gd\") pod \"65f92dec-a0b2-467b-9b90-c9b218ff556d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") "
Sep 10 13:45:56.998144 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.998053    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsz4h\" (UniqueName: \"kubernetes.io/projected/cfcfd673-94f5-4601-bb1e-e562075c207f-kube-api-access-gsz4h\") pod \"cfcfd673-94f5-4601-bb1e-e562075c207f\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") "
Sep 10 13:45:56.998189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.998177    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65f92dec-a0b2-467b-9b90-c9b218ff556d-plugins-config-volume\") pod \"65f92dec-a0b2-467b-9b90-c9b218ff556d\" (UID: \"65f92dec-a0b2-467b-9b90-c9b218ff556d\") "
Sep 10 13:45:56.998234 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.998221    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfcfd673-94f5-4601-bb1e-e562075c207f-plugins-config-volume\") pod \"cfcfd673-94f5-4601-bb1e-e562075c207f\" (UID: \"cfcfd673-94f5-4601-bb1e-e562075c207f\") "
Sep 10 13:45:56.998542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.998507    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f92dec-a0b2-467b-9b90-c9b218ff556d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65f92dec-a0b2-467b-9b90-c9b218ff556d" (UID: "65f92dec-a0b2-467b-9b90-c9b218ff556d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:45:56.998637 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:56.998574    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfcfd673-94f5-4601-bb1e-e562075c207f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cfcfd673-94f5-4601-bb1e-e562075c207f" (UID: "cfcfd673-94f5-4601-bb1e-e562075c207f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:45:57.000567 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.000538    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfcfd673-94f5-4601-bb1e-e562075c207f-kube-api-access-gsz4h" (OuterVolumeSpecName: "kube-api-access-gsz4h") pod "cfcfd673-94f5-4601-bb1e-e562075c207f" (UID: "cfcfd673-94f5-4601-bb1e-e562075c207f"). InnerVolumeSpecName "kube-api-access-gsz4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:57.000648 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.000571    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f92dec-a0b2-467b-9b90-c9b218ff556d-kube-api-access-wp9gd" (OuterVolumeSpecName: "kube-api-access-wp9gd") pod "65f92dec-a0b2-467b-9b90-c9b218ff556d" (UID: "65f92dec-a0b2-467b-9b90-c9b218ff556d"). InnerVolumeSpecName "kube-api-access-wp9gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:45:57.072176 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.072106    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:57.099635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.099610    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cfcfd673-94f5-4601-bb1e-e562075c207f-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:57.099635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.099634    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp9gd\" (UniqueName: \"kubernetes.io/projected/65f92dec-a0b2-467b-9b90-c9b218ff556d-kube-api-access-wp9gd\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:57.099798 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.099648    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsz4h\" (UniqueName: \"kubernetes.io/projected/cfcfd673-94f5-4601-bb1e-e562075c207f-kube-api-access-gsz4h\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:57.099798 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.099683    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65f92dec-a0b2-467b-9b90-c9b218ff556d-plugins-config-volume\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:45:57.248235 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.248197    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6ecdd9e-5f8b-498a-8a72-ed92da5d9338" path="/var/lib/kubelet/pods/c6ecdd9e-5f8b-498a-8a72-ed92da5d9338/volumes"
Sep 10 13:45:57.321395 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.321363    2551 generic.go:356] "Generic (PLEG): container finished" podID="cfcfd673-94f5-4601-bb1e-e562075c207f" containerID="7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6" exitCode=0
Sep 10 13:45:57.321511 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.321405    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"
Sep 10 13:45:57.321511 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.321441    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv" event={"ID":"cfcfd673-94f5-4601-bb1e-e562075c207f","Type":"ContainerDied","Data":"7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6"}
Sep 10 13:45:57.321511 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.321492    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv" event={"ID":"cfcfd673-94f5-4601-bb1e-e562075c207f","Type":"ContainerDied","Data":"a0e0c99c4d6d89687a06d4a5719756685c0e06f7a6e1def860223609b770c960"}
Sep 10 13:45:57.321640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.321514    2551 scope.go:117] "RemoveContainer" containerID="7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6"
Sep 10 13:45:57.322800 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.322780    2551 generic.go:356] "Generic (PLEG): container finished" podID="65f92dec-a0b2-467b-9b90-c9b218ff556d" containerID="41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc" exitCode=0
Sep 10 13:45:57.322889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.322809    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d" event={"ID":"65f92dec-a0b2-467b-9b90-c9b218ff556d","Type":"ContainerDied","Data":"41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc"}
Sep 10 13:45:57.322889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.322828    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"
Sep 10 13:45:57.322889 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.322855    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d" event={"ID":"65f92dec-a0b2-467b-9b90-c9b218ff556d","Type":"ContainerDied","Data":"5d548fe67488866e107d8a6e0d62332b7cbabe9d7743640a2ecc95638b3bb515"}
Sep 10 13:45:57.329511 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.329497    2551 scope.go:117] "RemoveContainer" containerID="7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6"
Sep 10 13:45:57.329803 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:45:57.329782    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6\": container with ID starting with 7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6 not found: ID does not exist" containerID="7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6"
Sep 10 13:45:57.329855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.329812    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6"} err="failed to get container status \"7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6\": rpc error: code = NotFound desc = could not find container \"7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6\": container with ID starting with 7c8908c5e3fc752ac1345dd616054708c8f9c99a5f6f4cbaa90a372e390d8cc6 not found: ID does not exist"
Sep 10 13:45:57.329855 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.329828    2551 scope.go:117] "RemoveContainer" containerID="41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc"
Sep 10 13:45:57.336707 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.336692    2551 scope.go:117] "RemoveContainer" containerID="41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc"
Sep 10 13:45:57.336939 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:45:57.336922    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc\": container with ID starting with 41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc not found: ID does not exist" containerID="41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc"
Sep 10 13:45:57.336996 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.336943    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc"} err="failed to get container status \"41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc\": rpc error: code = NotFound desc = could not find container \"41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc\": container with ID starting with 41e548163bc4fdf007aa6d1886d74239167ab4abf5ab291cd95bf4bfa3fe3dcc not found: ID does not exist"
Sep 10 13:45:57.342173 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.342153    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"]
Sep 10 13:45:57.346530 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.346512    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-7657ef-656979cd94-chg7d"]
Sep 10 13:45:57.367571 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.367551    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"]
Sep 10 13:45:57.372146 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.372128    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-7657ef-757c8d45c5-k7mjv"]
Sep 10 13:45:57.475064 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:57.475009    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:45:59.253434 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:59.253404    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65f92dec-a0b2-467b-9b90-c9b218ff556d" path="/var/lib/kubelet/pods/65f92dec-a0b2-467b-9b90-c9b218ff556d/volumes"
Sep 10 13:45:59.253801 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:59.253746    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfcfd673-94f5-4601-bb1e-e562075c207f" path="/var/lib/kubelet/pods/cfcfd673-94f5-4601-bb1e-e562075c207f/volumes"
Sep 10 13:45:59.790401 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:45:59.790371    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:46:05.228615 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:05.228583    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:05.352999 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:05.352967    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:05.671536 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:05.671498    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:06.085828 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.085794    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"]
Sep 10 13:46:06.086135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086122    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecfb5e47-4be3-405c-898e-a130cd5feeb2" containerName="cleanup"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086136    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecfb5e47-4be3-405c-898e-a130cd5feeb2" containerName="cleanup"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086152    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086157    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086164    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfcfd673-94f5-4601-bb1e-e562075c207f" containerName="payload-processing"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086170    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfcfd673-94f5-4601-bb1e-e562075c207f" containerName="payload-processing"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086175    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f92dec-a0b2-467b-9b90-c9b218ff556d" containerName="payload-pre-processing"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086180    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f92dec-a0b2-467b-9b90-c9b218ff556d" containerName="payload-pre-processing"
Sep 10 13:46:06.086183 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086187    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ecdd9e-5f8b-498a-8a72-ed92da5d9338" containerName="cleanup"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086192    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ecdd9e-5f8b-498a-8a72-ed92da5d9338" containerName="cleanup"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086198    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" containerName="payload-pre-processing"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086203    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" containerName="payload-pre-processing"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086212    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086218    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086271    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba16e5ef-5e5f-48cb-b2a7-7ac3e9535269" containerName="payload-pre-processing"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086287    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecfb5e47-4be3-405c-898e-a130cd5feeb2" containerName="cleanup"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086293    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ecdd9e-5f8b-498a-8a72-ed92da5d9338" containerName="cleanup"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086299    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfcfd673-94f5-4601-bb1e-e562075c207f" containerName="payload-processing"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086303    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086310    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c9f8baf-6a23-46cc-93ce-1c899a25988f" containerName="istio-proxy"
Sep 10 13:46:06.086460 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.086317    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f92dec-a0b2-467b-9b90-c9b218ff556d" containerName="payload-pre-processing"
Sep 10 13:46:06.090604 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.090587    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"
Sep 10 13:46:06.098313 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.098294    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:46:06.098598 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.098582    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wtblv\""
Sep 10 13:46:06.122578 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.122539    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"]
Sep 10 13:46:06.170081 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.170051    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d181230-3ab3-4134-ba45-ecbc1342d192-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz\" (UID: \"2d181230-3ab3-4134-ba45-ecbc1342d192\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"
Sep 10 13:46:06.271189 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.271153    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d181230-3ab3-4134-ba45-ecbc1342d192-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz\" (UID: \"2d181230-3ab3-4134-ba45-ecbc1342d192\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"
Sep 10 13:46:06.271778 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.271758    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d181230-3ab3-4134-ba45-ecbc1342d192-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz\" (UID: \"2d181230-3ab3-4134-ba45-ecbc1342d192\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"
Sep 10 13:46:06.346394 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.346266    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:06.400816 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.400783    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"
Sep 10 13:46:06.688822 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:06.688795    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"]
Sep 10 13:46:06.689817 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:46:06.689794    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d181230_3ab3_4134_ba45_ecbc1342d192.slice/crio-6b8526d0733af1807a97729dce604be6dea1fe6494d904417611a00b94919543 WatchSource:0}: Error finding container 6b8526d0733af1807a97729dce604be6dea1fe6494d904417611a00b94919543: Status 404 returned error can't find the container with id 6b8526d0733af1807a97729dce604be6dea1fe6494d904417611a00b94919543
Sep 10 13:46:07.370379 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:07.370352    2551 generic.go:356] "Generic (PLEG): container finished" podID="2d181230-3ab3-4134-ba45-ecbc1342d192" containerID="e4b59ef1d488b0291ea103cf088eecf76cd227f9b2ec256dab9042b9e98fa4b3" exitCode=0
Sep 10 13:46:07.370795 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:07.370421    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz" event={"ID":"2d181230-3ab3-4134-ba45-ecbc1342d192","Type":"ContainerDied","Data":"e4b59ef1d488b0291ea103cf088eecf76cd227f9b2ec256dab9042b9e98fa4b3"}
Sep 10 13:46:07.370795 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:07.370462    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz" event={"ID":"2d181230-3ab3-4134-ba45-ecbc1342d192","Type":"ContainerStarted","Data":"6b8526d0733af1807a97729dce604be6dea1fe6494d904417611a00b94919543"}
Sep 10 13:46:08.502808 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:08.502787    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"
Sep 10 13:46:08.590552 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:08.590529    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d181230-3ab3-4134-ba45-ecbc1342d192-maas-api-service-ca\") pod \"2d181230-3ab3-4134-ba45-ecbc1342d192\" (UID: \"2d181230-3ab3-4134-ba45-ecbc1342d192\") "
Sep 10 13:46:08.590852 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:08.590829    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d181230-3ab3-4134-ba45-ecbc1342d192-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d181230-3ab3-4134-ba45-ecbc1342d192" (UID: "2d181230-3ab3-4134-ba45-ecbc1342d192"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 13:46:08.691585 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:08.691511    2551 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d181230-3ab3-4134-ba45-ecbc1342d192-maas-api-service-ca\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:46:09.379636 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:09.379609    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"
Sep 10 13:46:09.379818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:09.379606    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz" event={"ID":"2d181230-3ab3-4134-ba45-ecbc1342d192","Type":"ContainerDied","Data":"6b8526d0733af1807a97729dce604be6dea1fe6494d904417611a00b94919543"}
Sep 10 13:46:09.379818 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:09.379745    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b8526d0733af1807a97729dce604be6dea1fe6494d904417611a00b94919543"
Sep 10 13:46:17.257081 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.257049    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:17.297407 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.297379    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"]
Sep 10 13:46:17.305541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.305519    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7739cb-29817465-5xj4m"]
Sep 10 13:46:17.439264 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.439230    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:17.534413 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.534381    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:17.800789 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.800707    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:17.858387 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.858350    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"]
Sep 10 13:46:17.864040 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.864014    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-7739cb-rnftz"]
Sep 10 13:46:17.882379 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:17.882356    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:18.379852 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:18.379815    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:19.247581 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:19.247547    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d181230-3ab3-4134-ba45-ecbc1342d192" path="/var/lib/kubelet/pods/2d181230-3ab3-4134-ba45-ecbc1342d192/volumes"
Sep 10 13:46:19.247852 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:19.247839    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecfb5e47-4be3-405c-898e-a130cd5feeb2" path="/var/lib/kubelet/pods/ecfb5e47-4be3-405c-898e-a130cd5feeb2/volumes"
Sep 10 13:46:19.933300 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:19.933266    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:46:26.389621 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:26.389587    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:26.489229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:26.489200    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:26.714821 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:26.714745    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:46:30.801198 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:46:30.801161    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:47:15.285277 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:47:15.285249    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:47:15.290083 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:47:15.290063    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:47:51.962358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:47:51.962321    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:47:52.906988 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:47:52.906952    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:47:53.303508 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:47:53.303478    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:47:54.871736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:47:54.871657    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:48:00.428223 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:00.428184    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:48:00.526268 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:00.526230    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:48:20.509149 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:20.509117    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:48:28.783590 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:28.783554    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:48:35.488322 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:35.488287    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:48:44.010499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:44.010410    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:48:48.906147 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:48.906087    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:48:58.690499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:48:58.690460    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:49:06.993311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:49:06.993276    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:49:15.592970 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:49:15.592935    2551 scope.go:117] "RemoveContainer" containerID="ddd0b692cdb26fcc7456cdd6b85ddbdf8d6bb1fb68892d0458309ce3d09112e8"
Sep 10 13:49:51.791086 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:49:51.791047    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:50:29.539468 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:29.539398    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:50:29.639001 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:29.638947    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:50:37.298529 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:37.298499    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:50:45.506984 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:45.506949    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:50:50.924897 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.924854    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"]
Sep 10 13:50:50.925391 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.925309    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d181230-3ab3-4134-ba45-ecbc1342d192" containerName="revoke-keys"
Sep 10 13:50:50.925391 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.925328    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d181230-3ab3-4134-ba45-ecbc1342d192" containerName="revoke-keys"
Sep 10 13:50:50.925513 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.925428    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d181230-3ab3-4134-ba45-ecbc1342d192" containerName="revoke-keys"
Sep 10 13:50:50.928388 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.928368    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:50.931517 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.931499    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-af9e9484-kserve-self-signed-certs\""
Sep 10 13:50:50.943070 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.943048    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"]
Sep 10 13:50:50.975006 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.974983    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:50:50.985246 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.985220    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-tmp-dir\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:50.985309 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.985251    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c9kb\" (UniqueName: \"kubernetes.io/projected/bb499c3d-b37b-44e6-b93f-517a811de67c-kube-api-access-2c9kb\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:50.985309 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.985274    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb499c3d-b37b-44e6-b93f-517a811de67c-tls-certs\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:50.985394 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.985351    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-home\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:50.985394 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.985385    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-dshm\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:50.985466 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:50.985402    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-model-cache\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.086467 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086439    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-dshm\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.086467 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086470    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-model-cache\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.086721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086522    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-tmp-dir\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.086721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086550    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9kb\" (UniqueName: \"kubernetes.io/projected/bb499c3d-b37b-44e6-b93f-517a811de67c-kube-api-access-2c9kb\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.086721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086589    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb499c3d-b37b-44e6-b93f-517a811de67c-tls-certs\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.086721 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086638    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-home\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.086970 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086947    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-model-cache\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.087034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.086957    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-tmp-dir\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.087034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.087015    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-home\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.088934 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.088900    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-dshm\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.089113 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.089095    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb499c3d-b37b-44e6-b93f-517a811de67c-tls-certs\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.103231 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.103201    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c9kb\" (UniqueName: \"kubernetes.io/projected/bb499c3d-b37b-44e6-b93f-517a811de67c-kube-api-access-2c9kb\") pod \"e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") " pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.237489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.237408    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:50:51.525736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.525712    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"]
Sep 10 13:50:51.527647 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:50:51.527625    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb499c3d_b37b_44e6_b93f_517a811de67c.slice/crio-cd666f5da603a34b5e23624a80965a4cf6dbaad8b3f493cf465d2056da882bbb WatchSource:0}: Error finding container cd666f5da603a34b5e23624a80965a4cf6dbaad8b3f493cf465d2056da882bbb: Status 404 returned error can't find the container with id cd666f5da603a34b5e23624a80965a4cf6dbaad8b3f493cf465d2056da882bbb
Sep 10 13:50:51.615438 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.615408    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc" event={"ID":"bb499c3d-b37b-44e6-b93f-517a811de67c","Type":"ContainerStarted","Data":"2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509"}
Sep 10 13:50:51.615618 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.615458    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc" event={"ID":"bb499c3d-b37b-44e6-b93f-517a811de67c","Type":"ContainerStarted","Data":"cd666f5da603a34b5e23624a80965a4cf6dbaad8b3f493cf465d2056da882bbb"}
Sep 10 13:50:51.637392 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:51.637326    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc" podStartSLOduration=1.637311477 podStartE2EDuration="1.637311477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:50:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:50:51.636277291 +0000 UTC m=+2016.847470548" watchObservedRunningTime="2026-09-10 13:50:51.637311477 +0000 UTC m=+2016.848504734"
Sep 10 13:50:52.683699 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:50:52.683653    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:51:01.237580 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:01.237536    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:51:01.238214 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:01.237606    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:51:01.251027 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:01.251004    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:51:01.671151 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:01.671120    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:51:14.704755 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.704720    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"]
Sep 10 13:51:14.708308 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.708291    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.711563 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.711544    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-af9e9484-kserve-self-signed-certs\""
Sep 10 13:51:14.720677 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.720642    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"]
Sep 10 13:51:14.792768 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.792730    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfql5\" (UniqueName: \"kubernetes.io/projected/0b0f1353-6feb-4046-bf1f-5c7e75113267-kube-api-access-kfql5\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.792925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.792809    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-dshm\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.792925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.792870    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-tmp-dir\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.793039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.792931    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-home\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.793039 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.792999    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-model-cache\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.793135 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.793038    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b0f1353-6feb-4046-bf1f-5c7e75113267-tls-certs\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.840748 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.840717    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:51:14.893733 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.893703    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-dshm\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.893885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.893752    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-tmp-dir\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.893885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.893784    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-home\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.893885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.893802    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-model-cache\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.893885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.893819    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b0f1353-6feb-4046-bf1f-5c7e75113267-tls-certs\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.893885 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.893840    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfql5\" (UniqueName: \"kubernetes.io/projected/0b0f1353-6feb-4046-bf1f-5c7e75113267-kube-api-access-kfql5\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.894164 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.894140    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-tmp-dir\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.894224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.894156    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-model-cache\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.894274 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.894259    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-home\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.896057 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.896036    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-dshm\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.896197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.896179    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b0f1353-6feb-4046-bf1f-5c7e75113267-tls-certs\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:14.912132 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:14.912108    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfql5\" (UniqueName: \"kubernetes.io/projected/0b0f1353-6feb-4046-bf1f-5c7e75113267-kube-api-access-kfql5\") pod \"e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") " pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:15.017801 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:15.017716    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:15.322977 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:15.322937    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"]
Sep 10 13:51:15.326342 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:51:15.326312    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b0f1353_6feb_4046_bf1f_5c7e75113267.slice/crio-652abd6d7910f8e376da85a5e0a6e97e3b6e18670c542ca18eeff80d73cbf39c WatchSource:0}: Error finding container 652abd6d7910f8e376da85a5e0a6e97e3b6e18670c542ca18eeff80d73cbf39c: Status 404 returned error can't find the container with id 652abd6d7910f8e376da85a5e0a6e97e3b6e18670c542ca18eeff80d73cbf39c
Sep 10 13:51:15.617043 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:15.616954    2551 scope.go:117] "RemoveContainer" containerID="191c6635cf3abf3ea1c285d5c1bbd56c907848c03620e465050325e897f7637c"
Sep 10 13:51:15.625074 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:15.625054    2551 scope.go:117] "RemoveContainer" containerID="fff0e84e8a12be832132b0ab32b19d614dbcb90e7ed64d681311e8b63f22ff8e"
Sep 10 13:51:15.724265 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:15.724231    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2" event={"ID":"0b0f1353-6feb-4046-bf1f-5c7e75113267","Type":"ContainerStarted","Data":"8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774"}
Sep 10 13:51:15.724678 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:15.724273    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2" event={"ID":"0b0f1353-6feb-4046-bf1f-5c7e75113267","Type":"ContainerStarted","Data":"652abd6d7910f8e376da85a5e0a6e97e3b6e18670c542ca18eeff80d73cbf39c"}
Sep 10 13:51:15.747500 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:15.747454    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2" podStartSLOduration=1.747439003 podStartE2EDuration="1.747439003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:51:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:51:15.746943207 +0000 UTC m=+2040.958136464" watchObservedRunningTime="2026-09-10 13:51:15.747439003 +0000 UTC m=+2040.958632260"
Sep 10 13:51:17.901283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:17.901243    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:51:25.018366 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:25.018315    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:25.018366 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:25.018374    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:25.030640 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:25.030617    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:25.780876 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:25.780846    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:51:49.657175 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:49.657136    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"]
Sep 10 13:51:49.672636 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:49.672610    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:51:51.502723 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:51.502684    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"]
Sep 10 13:51:51.520554 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:51.520521    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:51:51.887252 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:51:51.887218    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:52:00.936856 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:00.936818    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:52:04.676386 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.676322    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc" podUID="bb499c3d-b37b-44e6-b93f-517a811de67c" containerName="main" containerID="cri-o://2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509" gracePeriod=45
Sep 10 13:52:04.918922 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.918896    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:52:04.944026 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.943940    2551 generic.go:356] "Generic (PLEG): container finished" podID="bb499c3d-b37b-44e6-b93f-517a811de67c" containerID="2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509" exitCode=0
Sep 10 13:52:04.944026 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.943984    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc" event={"ID":"bb499c3d-b37b-44e6-b93f-517a811de67c","Type":"ContainerDied","Data":"2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509"}
Sep 10 13:52:04.944026 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.944015    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc" event={"ID":"bb499c3d-b37b-44e6-b93f-517a811de67c","Type":"ContainerDied","Data":"cd666f5da603a34b5e23624a80965a4cf6dbaad8b3f493cf465d2056da882bbb"}
Sep 10 13:52:04.944270 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.944033    2551 scope.go:117] "RemoveContainer" containerID="2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509"
Sep 10 13:52:04.944270 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.943986    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"
Sep 10 13:52:04.951845 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.951828    2551 scope.go:117] "RemoveContainer" containerID="2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509"
Sep 10 13:52:04.952121 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:52:04.952105    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509\": container with ID starting with 2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509 not found: ID does not exist" containerID="2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509"
Sep 10 13:52:04.952164 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:04.952129    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509"} err="failed to get container status \"2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509\": rpc error: code = NotFound desc = could not find container \"2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509\": container with ID starting with 2015ff54a96fb0425e4a13b23e143ae694eacbc9f5fdfbe278b3c8a03f7b3509 not found: ID does not exist"
Sep 10 13:52:05.030802 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.030762    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-home\") pod \"bb499c3d-b37b-44e6-b93f-517a811de67c\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") "
Sep 10 13:52:05.031008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.030823    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c9kb\" (UniqueName: \"kubernetes.io/projected/bb499c3d-b37b-44e6-b93f-517a811de67c-kube-api-access-2c9kb\") pod \"bb499c3d-b37b-44e6-b93f-517a811de67c\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") "
Sep 10 13:52:05.031008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.030855    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb499c3d-b37b-44e6-b93f-517a811de67c-tls-certs\") pod \"bb499c3d-b37b-44e6-b93f-517a811de67c\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") "
Sep 10 13:52:05.031008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.030884    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-model-cache\") pod \"bb499c3d-b37b-44e6-b93f-517a811de67c\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") "
Sep 10 13:52:05.031008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.030926    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-tmp-dir\") pod \"bb499c3d-b37b-44e6-b93f-517a811de67c\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") "
Sep 10 13:52:05.031008 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.030959    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-dshm\") pod \"bb499c3d-b37b-44e6-b93f-517a811de67c\" (UID: \"bb499c3d-b37b-44e6-b93f-517a811de67c\") "
Sep 10 13:52:05.031283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.031058    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-home" (OuterVolumeSpecName: "home") pod "bb499c3d-b37b-44e6-b93f-517a811de67c" (UID: "bb499c3d-b37b-44e6-b93f-517a811de67c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:05.031283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.031213    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:05.031283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.031182    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-model-cache" (OuterVolumeSpecName: "model-cache") pod "bb499c3d-b37b-44e6-b93f-517a811de67c" (UID: "bb499c3d-b37b-44e6-b93f-517a811de67c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:05.031283 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.031244    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bb499c3d-b37b-44e6-b93f-517a811de67c" (UID: "bb499c3d-b37b-44e6-b93f-517a811de67c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:05.033095 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.033067    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-dshm" (OuterVolumeSpecName: "dshm") pod "bb499c3d-b37b-44e6-b93f-517a811de67c" (UID: "bb499c3d-b37b-44e6-b93f-517a811de67c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:05.033224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.033114    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb499c3d-b37b-44e6-b93f-517a811de67c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bb499c3d-b37b-44e6-b93f-517a811de67c" (UID: "bb499c3d-b37b-44e6-b93f-517a811de67c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:52:05.033224 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.033142    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb499c3d-b37b-44e6-b93f-517a811de67c-kube-api-access-2c9kb" (OuterVolumeSpecName: "kube-api-access-2c9kb") pod "bb499c3d-b37b-44e6-b93f-517a811de67c" (UID: "bb499c3d-b37b-44e6-b93f-517a811de67c"). InnerVolumeSpecName "kube-api-access-2c9kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:52:05.132285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.132244    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c9kb\" (UniqueName: \"kubernetes.io/projected/bb499c3d-b37b-44e6-b93f-517a811de67c-kube-api-access-2c9kb\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:05.132285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.132279    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb499c3d-b37b-44e6-b93f-517a811de67c-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:05.132495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.132289    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:05.132495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.132312    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:05.132495 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.132320    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb499c3d-b37b-44e6-b93f-517a811de67c-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:05.273499 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.273463    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"]
Sep 10 13:52:05.282812 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:05.282775    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-af9e9484-kserve-869c4d788c-8jpbc"]
Sep 10 13:52:06.522917 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.522875    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2" podUID="0b0f1353-6feb-4046-bf1f-5c7e75113267" containerName="main" containerID="cri-o://8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774" gracePeriod=45
Sep 10 13:52:06.768745 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.768717    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:52:06.847240 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847154    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-home\") pod \"0b0f1353-6feb-4046-bf1f-5c7e75113267\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") "
Sep 10 13:52:06.847240 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847211    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-tmp-dir\") pod \"0b0f1353-6feb-4046-bf1f-5c7e75113267\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") "
Sep 10 13:52:06.847489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847246    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-model-cache\") pod \"0b0f1353-6feb-4046-bf1f-5c7e75113267\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") "
Sep 10 13:52:06.847489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847278    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfql5\" (UniqueName: \"kubernetes.io/projected/0b0f1353-6feb-4046-bf1f-5c7e75113267-kube-api-access-kfql5\") pod \"0b0f1353-6feb-4046-bf1f-5c7e75113267\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") "
Sep 10 13:52:06.847489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847331    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b0f1353-6feb-4046-bf1f-5c7e75113267-tls-certs\") pod \"0b0f1353-6feb-4046-bf1f-5c7e75113267\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") "
Sep 10 13:52:06.847489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847351    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-dshm\") pod \"0b0f1353-6feb-4046-bf1f-5c7e75113267\" (UID: \"0b0f1353-6feb-4046-bf1f-5c7e75113267\") "
Sep 10 13:52:06.847489 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847453    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-home" (OuterVolumeSpecName: "home") pod "0b0f1353-6feb-4046-bf1f-5c7e75113267" (UID: "0b0f1353-6feb-4046-bf1f-5c7e75113267"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:06.847722 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847540    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0b0f1353-6feb-4046-bf1f-5c7e75113267" (UID: "0b0f1353-6feb-4046-bf1f-5c7e75113267"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:06.847722 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847554    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-model-cache" (OuterVolumeSpecName: "model-cache") pod "0b0f1353-6feb-4046-bf1f-5c7e75113267" (UID: "0b0f1353-6feb-4046-bf1f-5c7e75113267"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:06.847722 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.847598    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-home\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:06.849542 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.849515    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0f1353-6feb-4046-bf1f-5c7e75113267-kube-api-access-kfql5" (OuterVolumeSpecName: "kube-api-access-kfql5") pod "0b0f1353-6feb-4046-bf1f-5c7e75113267" (UID: "0b0f1353-6feb-4046-bf1f-5c7e75113267"). InnerVolumeSpecName "kube-api-access-kfql5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 13:52:06.849652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.849549    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-dshm" (OuterVolumeSpecName: "dshm") pod "0b0f1353-6feb-4046-bf1f-5c7e75113267" (UID: "0b0f1353-6feb-4046-bf1f-5c7e75113267"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 13:52:06.849652 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.849555    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b0f1353-6feb-4046-bf1f-5c7e75113267-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0b0f1353-6feb-4046-bf1f-5c7e75113267" (UID: "0b0f1353-6feb-4046-bf1f-5c7e75113267"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 13:52:06.948860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.948807    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-tmp-dir\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:06.948860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.948851    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-model-cache\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:06.948860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.948866    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfql5\" (UniqueName: \"kubernetes.io/projected/0b0f1353-6feb-4046-bf1f-5c7e75113267-kube-api-access-kfql5\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:06.948860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.948873    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b0f1353-6feb-4046-bf1f-5c7e75113267-tls-certs\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:06.948860 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.948883    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b0f1353-6feb-4046-bf1f-5c7e75113267-dshm\") on node \"ip-10-0-137-9.ec2.internal\" DevicePath \"\""
Sep 10 13:52:06.954469 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.954442    2551 generic.go:356] "Generic (PLEG): container finished" podID="0b0f1353-6feb-4046-bf1f-5c7e75113267" containerID="8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774" exitCode=0
Sep 10 13:52:06.954626 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.954484    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"
Sep 10 13:52:06.954626 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.954524    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2" event={"ID":"0b0f1353-6feb-4046-bf1f-5c7e75113267","Type":"ContainerDied","Data":"8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774"}
Sep 10 13:52:06.954626 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.954569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2" event={"ID":"0b0f1353-6feb-4046-bf1f-5c7e75113267","Type":"ContainerDied","Data":"652abd6d7910f8e376da85a5e0a6e97e3b6e18670c542ca18eeff80d73cbf39c"}
Sep 10 13:52:06.954626 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.954591    2551 scope.go:117] "RemoveContainer" containerID="8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774"
Sep 10 13:52:06.963389 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.963372    2551 scope.go:117] "RemoveContainer" containerID="8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774"
Sep 10 13:52:06.963706 ip-10-0-137-9 kubenswrapper[2551]: E0910 13:52:06.963659    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774\": container with ID starting with 8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774 not found: ID does not exist" containerID="8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774"
Sep 10 13:52:06.963763 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.963718    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774"} err="failed to get container status \"8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774\": rpc error: code = NotFound desc = could not find container \"8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774\": container with ID starting with 8305c5a86e8a65c110a44b07c3fefd4dc79a002122ece0fabd9ad6f696e0e774 not found: ID does not exist"
Sep 10 13:52:06.983628 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.983598    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"]
Sep 10 13:52:06.994708 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:06.994683    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-af9e9484-kserve-787d96f967-ltrw2"]
Sep 10 13:52:07.248286 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:07.248207    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b0f1353-6feb-4046-bf1f-5c7e75113267" path="/var/lib/kubelet/pods/0b0f1353-6feb-4046-bf1f-5c7e75113267/volumes"
Sep 10 13:52:07.248591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:07.248579    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb499c3d-b37b-44e6-b93f-517a811de67c" path="/var/lib/kubelet/pods/bb499c3d-b37b-44e6-b93f-517a811de67c/volumes"
Sep 10 13:52:10.743257 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:10.743222    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:52:15.310971 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:15.310941    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:52:15.317141 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:15.317123    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z6jq7_5c7a8678-8fbf-44ad-ba97-dda31a9d9e5d/ovn-acl-logging/0.log"
Sep 10 13:52:15.646365 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:15.646279    2551 scope.go:117] "RemoveContainer" containerID="e4b59ef1d488b0291ea103cf088eecf76cd227f9b2ec256dab9042b9e98fa4b3"
Sep 10 13:52:19.007148 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.007118    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:52:19.210850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.210814    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"]
Sep 10 13:52:19.211131 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.211120    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb499c3d-b37b-44e6-b93f-517a811de67c" containerName="main"
Sep 10 13:52:19.211180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.211133    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb499c3d-b37b-44e6-b93f-517a811de67c" containerName="main"
Sep 10 13:52:19.211180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.211148    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b0f1353-6feb-4046-bf1f-5c7e75113267" containerName="main"
Sep 10 13:52:19.211180 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.211154    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b0f1353-6feb-4046-bf1f-5c7e75113267" containerName="main"
Sep 10 13:52:19.211277 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.211202    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b0f1353-6feb-4046-bf1f-5c7e75113267" containerName="main"
Sep 10 13:52:19.211277 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.211212    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb499c3d-b37b-44e6-b93f-517a811de67c" containerName="main"
Sep 10 13:52:19.215344 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.215323    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.220772 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.220750    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 13:52:19.220907 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.220753    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 13:52:19.220907 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.220846    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 13:52:19.220907 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.220782    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5fn9n\""
Sep 10 13:52:19.220907 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.220887    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 13:52:19.227497 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.227476    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"]
Sep 10 13:52:19.255751 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.255731    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d52db69-2634-49ec-a75b-8d6a15044108-openshift-service-ca\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.255876 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.255856    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d52db69-2634-49ec-a75b-8d6a15044108-metrics-certs\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.255933 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.255900    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bxhw\" (UniqueName: \"kubernetes.io/projected/1d52db69-2634-49ec-a75b-8d6a15044108-kube-api-access-8bxhw\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.256001 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.255977    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d52db69-2634-49ec-a75b-8d6a15044108-maas-api-tls\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.357218 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.357192    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d52db69-2634-49ec-a75b-8d6a15044108-metrics-certs\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.357343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.357224    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bxhw\" (UniqueName: \"kubernetes.io/projected/1d52db69-2634-49ec-a75b-8d6a15044108-kube-api-access-8bxhw\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.357343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.357246    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d52db69-2634-49ec-a75b-8d6a15044108-maas-api-tls\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.357343 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.357304    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d52db69-2634-49ec-a75b-8d6a15044108-openshift-service-ca\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.357979 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.357955    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d52db69-2634-49ec-a75b-8d6a15044108-openshift-service-ca\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.359620 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.359599    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d52db69-2634-49ec-a75b-8d6a15044108-metrics-certs\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.359620 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.359612    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d52db69-2634-49ec-a75b-8d6a15044108-maas-api-tls\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.367130 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.367109    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bxhw\" (UniqueName: \"kubernetes.io/projected/1d52db69-2634-49ec-a75b-8d6a15044108-kube-api-access-8bxhw\") pod \"maas-api-584764ddc7-xcfv7\" (UID: \"1d52db69-2634-49ec-a75b-8d6a15044108\") " pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.524736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.524703    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:19.800199 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.800175    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"]
Sep 10 13:52:19.802236 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:52:19.802201    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d52db69_2634_49ec_a75b_8d6a15044108.slice/crio-046dd187a9fa27237c4e96170e88a3882536bc91e302222962a0c6f8d1e96ebd WatchSource:0}: Error finding container 046dd187a9fa27237c4e96170e88a3882536bc91e302222962a0c6f8d1e96ebd: Status 404 returned error can't find the container with id 046dd187a9fa27237c4e96170e88a3882536bc91e302222962a0c6f8d1e96ebd
Sep 10 13:52:19.803875 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:19.803853    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 13:52:20.014911 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:20.014877    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7" event={"ID":"1d52db69-2634-49ec-a75b-8d6a15044108","Type":"ContainerStarted","Data":"046dd187a9fa27237c4e96170e88a3882536bc91e302222962a0c6f8d1e96ebd"}
Sep 10 13:52:23.031187 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:23.031157    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7" event={"ID":"1d52db69-2634-49ec-a75b-8d6a15044108","Type":"ContainerStarted","Data":"d5bc4443fbb22feb7dd9f1d857c92b4993110d8c6bd46fe5ea907433708be17c"}
Sep 10 13:52:23.031531 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:23.031294    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:23.057680 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:23.057613    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7" podStartSLOduration=1.7627740840000001 podStartE2EDuration="4.05759818s" totalImagesPullingTime="2.294824096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 13:52:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:52:23.056784619 +0000 UTC m=+2108.267977873" watchObservedRunningTime="2026-09-10 13:52:23.05759818 +0000 UTC m=+2108.268791431"
Sep 10 13:52:26.162284 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:26.162254    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:52:29.040220 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:29.040183    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-584764ddc7-xcfv7"
Sep 10 13:52:34.397344 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:34.397304    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:52:40.021659 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:40.021625    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-qlmdf"]
Sep 10 13:52:42.414616 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:42.414586    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f8d55b679-mnnjk_f0b01acd-281c-45f7-b935-54a0ed20fa0e/authorino/0.log"
Sep 10 13:52:48.675217 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:48.675182    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fsmzw_f7c9e53d-7df2-4098-acf7-3b1340a4cba4/manager/0.log"
Sep 10 13:52:49.475591 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:49.475563    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699db476d-p6hwj_a59d01b6-6020-4c97-aba9-d97700e632d4/copy-manifests/0.log"
Sep 10 13:52:49.495158 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:49.495130    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699db476d-p6hwj_a59d01b6-6020-4c97-aba9-d97700e632d4/manager/0.log"
Sep 10 13:52:49.991229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:49.991193    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-584764ddc7-xcfv7_1d52db69-2634-49ec-a75b-8d6a15044108/maas-api/0.log"
Sep 10 13:52:50.116248 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:50.116214    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817450-jhxbl_02e41435-71ff-4706-8d6f-dc51e2fb117a/cleanup/0.log"
Sep 10 13:52:51.209635 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:51.209604    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f8d55b679-mnnjk_f0b01acd-281c-45f7-b935-54a0ed20fa0e/authorino/0.log"
Sep 10 13:52:51.588101 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:51.588071    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-plmr6_5496b84d-b738-4751-add2-fc9eafbb4040/kuadrant-console-plugin/0.log"
Sep 10 13:52:51.852315 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:51.852232    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qlmdf_b342d82d-efe0-449b-87b8-020f7dad5bbe/limitador/0.log"
Sep 10 13:52:52.739053 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:52.739025    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69d5c84f88-m94bf_4fd220db-a802-4fe8-aaa5-2e9da1b476bf/kube-auth-proxy/0.log"
Sep 10 13:52:54.250028 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:54.249997    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-jg5rb_ba75e13a-05ee-4b30-ae1f-d50f5dfcaf60/main/0.log"
Sep 10 13:52:54.375925 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:52:54.375896    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lb958_be1370a7-30c7-4864-aecb-799fd45f3f8d/main/0.log"
Sep 10 13:53:00.658736 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:00.658693    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:53:02.937778 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:02.937740    2551 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log" limit=1024
Sep 10 13:53:03.943305 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:03.943277    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8xc7t_8fe469d7-9e50-4779-b9f0-e92eb262acb6/global-pull-secret-syncer/0.log"
Sep 10 13:53:04.163529 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:04.163501    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dkrjp_98a61716-28c9-49e3-9121-9bbac2e21f08/konnectivity-agent/0.log"
Sep 10 13:53:04.255938 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:04.255865    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-9.ec2.internal_169e0ba168d588086edfb9d96364eff8/haproxy/0.log"
Sep 10 13:53:08.558195 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:08.558164    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-f8d55b679-mnnjk_f0b01acd-281c-45f7-b935-54a0ed20fa0e/authorino/0.log"
Sep 10 13:53:08.688241 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:08.688175    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-plmr6_5496b84d-b738-4751-add2-fc9eafbb4040/kuadrant-console-plugin/0.log"
Sep 10 13:53:08.773588 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:08.773564    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-qlmdf_b342d82d-efe0-449b-87b8-020f7dad5bbe/limitador/0.log"
Sep 10 13:53:10.997160 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:10.997133    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-tzw2c_01f5a2e6-19c5-442a-a515-1bf2464dfae6/monitoring-plugin/0.log"
Sep 10 13:53:11.036457 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.036427    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c97gc_4931e55d-5857-4a71-8239-3ebd0f5a6647/node-exporter/0.log"
Sep 10 13:53:11.064541 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.064517    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c97gc_4931e55d-5857-4a71-8239-3ebd0f5a6647/kube-rbac-proxy/0.log"
Sep 10 13:53:11.095318 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.095296    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c97gc_4931e55d-5857-4a71-8239-3ebd0f5a6647/init-textfile/0.log"
Sep 10 13:53:11.532899 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.532870    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb/prometheus/0.log"
Sep 10 13:53:11.584216 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.584143    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb/config-reloader/0.log"
Sep 10 13:53:11.622653 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.622624    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb/thanos-sidecar/0.log"
Sep 10 13:53:11.650507 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.650481    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb/kube-rbac-proxy-web/0.log"
Sep 10 13:53:11.680745 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.680722    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb/kube-rbac-proxy/0.log"
Sep 10 13:53:11.723515 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.723486    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb/kube-rbac-proxy-thanos/0.log"
Sep 10 13:53:11.751709 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:11.751686    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_223e3a49-1ab8-4b3d-a7aa-1b97d92aeadb/init-config-reloader/0.log"
Sep 10 13:53:12.103388 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:12.103361    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4584d64c-qfslj_50bd353d-50de-4220-b161-b6b76e0611dd/thanos-query/0.log"
Sep 10 13:53:12.132102 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:12.132080    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4584d64c-qfslj_50bd353d-50de-4220-b161-b6b76e0611dd/kube-rbac-proxy-web/0.log"
Sep 10 13:53:12.157946 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:12.157918    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4584d64c-qfslj_50bd353d-50de-4220-b161-b6b76e0611dd/kube-rbac-proxy/0.log"
Sep 10 13:53:12.185884 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:12.185811    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4584d64c-qfslj_50bd353d-50de-4220-b161-b6b76e0611dd/prom-label-proxy/0.log"
Sep 10 13:53:12.215311 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:12.215284    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4584d64c-qfslj_50bd353d-50de-4220-b161-b6b76e0611dd/kube-rbac-proxy-rules/0.log"
Sep 10 13:53:12.240994 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:12.240970    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4584d64c-qfslj_50bd353d-50de-4220-b161-b6b76e0611dd/kube-rbac-proxy-metrics/0.log"
Sep 10 13:53:14.172229 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.172198    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"]
Sep 10 13:53:14.179074 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.179044    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.195614 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.195584    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9tqt4\"/\"kube-root-ca.crt\""
Sep 10 13:53:14.195614 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.195602    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9tqt4\"/\"openshift-service-ca.crt\""
Sep 10 13:53:14.195868 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.195629    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9tqt4\"/\"default-dockercfg-hkk72\""
Sep 10 13:53:14.198762 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.198735    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"]
Sep 10 13:53:14.210921 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.210888    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-proc\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.211097 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.210936    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-lib-modules\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.211097 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.210955    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-podres\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.211097 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.210991    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-sys\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.211097 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.211081    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5swns\" (UniqueName: \"kubernetes.io/projected/d8d8ad7d-e615-4ba3-9177-0a4715902de6-kube-api-access-5swns\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311614 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311568    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-proc\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311623    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-lib-modules\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311646    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-podres\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311727    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-proc\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311745    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-sys\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311797    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5swns\" (UniqueName: \"kubernetes.io/projected/d8d8ad7d-e615-4ba3-9177-0a4715902de6-kube-api-access-5swns\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311815    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-podres\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.311850 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311819    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-lib-modules\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.312172 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.311869    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8d8ad7d-e615-4ba3-9177-0a4715902de6-sys\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.323143 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.323113    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5swns\" (UniqueName: \"kubernetes.io/projected/d8d8ad7d-e615-4ba3-9177-0a4715902de6-kube-api-access-5swns\") pod \"perf-node-gather-daemonset-mm5x9\" (UID: \"d8d8ad7d-e615-4ba3-9177-0a4715902de6\") " pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.489003 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.488922    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:14.767875 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:14.767725    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"]
Sep 10 13:53:14.770862 ip-10-0-137-9 kubenswrapper[2551]: W0910 13:53:14.770827    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd8d8ad7d_e615_4ba3_9177_0a4715902de6.slice/crio-15ba651cbfa37158e1e3312a04bc6783f6606fcc89e86204306052aa61ea9af0 WatchSource:0}: Error finding container 15ba651cbfa37158e1e3312a04bc6783f6606fcc89e86204306052aa61ea9af0: Status 404 returned error can't find the container with id 15ba651cbfa37158e1e3312a04bc6783f6606fcc89e86204306052aa61ea9af0
Sep 10 13:53:15.260362 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:15.260327    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9" event={"ID":"d8d8ad7d-e615-4ba3-9177-0a4715902de6","Type":"ContainerStarted","Data":"e0773120c8cf20336b514ad4ab8f4797c629ea102fc2046debe00fb6e42bf891"}
Sep 10 13:53:15.260801 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:15.260376    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9" event={"ID":"d8d8ad7d-e615-4ba3-9177-0a4715902de6","Type":"ContainerStarted","Data":"15ba651cbfa37158e1e3312a04bc6783f6606fcc89e86204306052aa61ea9af0"}
Sep 10 13:53:15.260801 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:15.260467    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:15.298468 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:15.298415    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9" podStartSLOduration=1.298396243 podStartE2EDuration="1.298396243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 13:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 13:53:15.29689819 +0000 UTC m=+2160.508091447" watchObservedRunningTime="2026-09-10 13:53:15.298396243 +0000 UTC m=+2160.509589537"
Sep 10 13:53:18.197467 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:18.197435    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wdq7m_9748123d-df56-462f-a50f-6bfd6d565527/dns/0.log"
Sep 10 13:53:18.252962 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:18.252936    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wdq7m_9748123d-df56-462f-a50f-6bfd6d565527/kube-rbac-proxy/0.log"
Sep 10 13:53:18.346679 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:18.346643    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lp4ht_b578b265-6bda-4a01-b349-e0fbf48b6065/dns-node-resolver/0.log"
Sep 10 13:53:18.985010 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:18.984979    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7fc87d7f97-48xl2_2bb7f1fa-465a-43b3-8884-962f45148f07/registry/0.log"
Sep 10 13:53:19.059197 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:19.059168    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s8c85_37d43d0f-edb7-4379-994a-c64fb9c09f31/node-ca/0.log"
Sep 10 13:53:20.748179 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:20.748145    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69d5c84f88-m94bf_4fd220db-a802-4fe8-aaa5-2e9da1b476bf/kube-auth-proxy/0.log"
Sep 10 13:53:21.274759 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:21.274733    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9tqt4/perf-node-gather-daemonset-mm5x9"
Sep 10 13:53:22.042252 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:22.042225    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mfbgb_0d3bc038-4830-456d-b704-ddedf5e5d2d2/serve-healthcheck-canary/0.log"
Sep 10 13:53:23.018017 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:23.017985    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gbsvc_e4236150-889c-4505-aade-ec24ea75d092/kube-rbac-proxy/0.log"
Sep 10 13:53:23.050165 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:23.050137    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gbsvc_e4236150-889c-4505-aade-ec24ea75d092/exporter/0.log"
Sep 10 13:53:23.076358 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:23.076294    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gbsvc_e4236150-889c-4505-aade-ec24ea75d092/extractor/0.log"
Sep 10 13:53:25.493939 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:25.493906    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69d5c84f88-m94bf_4fd220db-a802-4fe8-aaa5-2e9da1b476bf/kube-auth-proxy/0.log"
Sep 10 13:53:26.074473 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:26.074433    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-fsmzw_f7c9e53d-7df2-4098-acf7-3b1340a4cba4/manager/0.log"
Sep 10 13:53:26.578034 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:26.578005    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699db476d-p6hwj_a59d01b6-6020-4c97-aba9-d97700e632d4/manager/0.log"
Sep 10 13:53:26.613714 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:26.613685    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-699db476d-p6hwj_a59d01b6-6020-4c97-aba9-d97700e632d4/copy-manifests/0.log"
Sep 10 13:53:27.418285 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:27.418258    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69d5c84f88-m94bf_4fd220db-a802-4fe8-aaa5-2e9da1b476bf/kube-auth-proxy/0.log"
Sep 10 13:53:27.906124 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:27.906093    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78dbbc87c8-tgcj2_8c182f4d-0e12-4262-935a-75ee27cd3de0/manager/0.log"
Sep 10 13:53:27.938792 ip-10-0-137-9 kubenswrapper[2551]: I0910 13:53:27.938765    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-qt4jg_70f64087-b71f-4387-b2aa-2b13186e318b/openshift-lws-operator/0.log"