Sep 10 20:56:49.112302 ip-10-0-140-211 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 20:56:49.112317 ip-10-0-140-211 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 20:56:49.112327 ip-10-0-140-211 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 20:56:49.112650 ip-10-0-140-211 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 20:56:59.361572 ip-10-0-140-211 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 20:56:59.361590 ip-10-0-140-211 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b83a29dcf9774369b5d169c271d9ce12 --
Sep 10 20:58:38.555244 ip-10-0-140-211 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 20:58:38.894467 ip-10-0-140-211 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:38.894467 ip-10-0-140-211 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 20:58:38.894467 ip-10-0-140-211 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:38.894467 ip-10-0-140-211 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 20:58:38.894467 ip-10-0-140-211 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 20:58:38.896762 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.896680    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 20:58:38.899745 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899730    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:38.899745 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899745    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899749    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899752    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899755    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899758    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899761    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899764    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899767    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899769    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899771    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899774    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899777    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899781    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899785    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899789    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899792    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899795    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899798    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899801    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899804    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:38.899807 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899806    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899809    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899812    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899814    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899817    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899820    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899823    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899825    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899828    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899830    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899833    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899837    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899841    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899844    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899847    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899850    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899852    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899855    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899858    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:38.900300 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899861    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899864    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899867    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899869    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899872    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899874    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899877    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899879    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899882    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899885    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899887    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899890    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899892    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899895    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899898    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899901    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899904    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899906    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899909    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899911    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:38.900766 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899914    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899916    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899920    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899923    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899925    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899928    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899931    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899933    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899936    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899938    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899941    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899944    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899946    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899949    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899953    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899956    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899959    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899962    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899965    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:38.901276 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899982    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899984    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899987    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899989    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899993    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.899996    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900000    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900003    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900008    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900010    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900013    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900016    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900018    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900021    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900023    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900026    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900029    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900032    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900035    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:38.901765 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900037    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900723    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900730    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900733    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900736    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900739    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900743    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900747    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900749    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900752    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900755    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900759    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900763    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900766    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900769    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900772    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900775    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900778    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900781    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:38.902252 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900783    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900787    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900790    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900793    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900795    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900798    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900800    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900803    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900805    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900808    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900810    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900813    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900816    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900818    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900820    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900823    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900826    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900828    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900831    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900833    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:38.902722 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900836    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900839    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900842    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900845    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900847    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900849    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900852    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900854    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900857    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900860    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900862    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900865    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900867    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900870    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900873    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900876    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900878    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900881    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900883    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900886    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:38.903227 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900888    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900891    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900893    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900896    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900898    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900901    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900903    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900906    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900908    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900911    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900913    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900916    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900918    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900921    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900924    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900927    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900929    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900932    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900935    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900937    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:38.903730 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900941    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900944    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900946    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900949    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900952    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900954    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900957    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900960    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900963    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900982    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900985    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900988    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900990    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.900993    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901000    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901004    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901008    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901011    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901014    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:38.904232 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901017    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901020    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901410    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901415    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901418    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901421    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901425    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901428    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901430    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901436    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901438    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901441    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901443    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901446    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901449    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901452    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901454    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901457    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901460    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901462    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:38.904702 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901465    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901467    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901470    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901472    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901475    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901477    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901480    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901482    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901485    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901487    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901490    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901493    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901497    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901500    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901503    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901506    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901509    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901513    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901516    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901522    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:38.905201 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901524    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901529    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901531    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901534    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901537    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901539    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901542    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901544    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901546    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901549    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901551    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901554    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901556    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901559    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901562    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901564    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901567    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901569    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901572    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901574    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:38.905693 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901577    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901579    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901581    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901584    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901586    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901589    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901591    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901594    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901597    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901599    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901602    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901604    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901607    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901611    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901614    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901616    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901619    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901621    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901626    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:38.906205 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901629    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901632    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901635    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901637    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901641    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901644    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901647    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901650    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901653    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901655    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901658    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901660    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901663    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901665    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901668    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901670    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901673    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901675    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901678    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901681    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:38.906716 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901683    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.901686    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901750    2557 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901785    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901791    2557 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901795    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901801    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901804    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901808    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901812    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901815    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901818    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901821    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901825    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901828    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901831    2557 flags.go:64] FLAG: --cgroup-root=""
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901834    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901837    2557 flags.go:64] FLAG: --client-ca-file=""
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901840    2557 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901842    2557 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901846    2557 flags.go:64] FLAG: --cluster-domain=""
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901849    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901851    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 20:58:38.907218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901855    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901858    2557 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901866    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901871    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901874    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901878    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901881    2557 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901884    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901887    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901890    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901893    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901896    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901899    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901902    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901905    2557 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901909    2557 flags.go:64] FLAG: --enable-server="true"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901913    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901917    2557 flags.go:64] FLAG: --event-burst="100"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901920    2557 flags.go:64] FLAG: --event-qps="50"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901923    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901926    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901929    2557 flags.go:64] FLAG: --eviction-hard=""
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901932    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901935    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901938    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 20:58:38.907796 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901941    2557 flags.go:64] FLAG: --eviction-soft=""
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901944    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901947    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901950    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901953    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901955    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901958    2557 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901961    2557 flags.go:64] FLAG: --feature-gates=""
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901965    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901981    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901986    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901989    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901992    2557 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901995    2557 flags.go:64] FLAG: --help="false"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.901998    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-140-211.ec2.internal"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902001    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902004    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902007    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902010    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902013    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902016    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902019    2557 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902023    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 20:58:38.908423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902026    2557 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902030    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902033    2557 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902035    2557 flags.go:64] FLAG: --kube-reserved=""
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902038    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902041    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902044    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902047    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902050    2557 flags.go:64] FLAG: --lock-file=""
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902052    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902055    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902058    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902063    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902066    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902069    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902071    2557 flags.go:64] FLAG: --logging-format="text"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902074    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902078    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902081    2557 flags.go:64] FLAG: --manifest-url=""
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902085    2557 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902089    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902092    2557 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902096    2557 flags.go:64] FLAG: --max-pods="110"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902099    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902101    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 20:58:38.909061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902104    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902107    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902110    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902113    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902116    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902123    2557 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902127    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902130    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902133    2557 flags.go:64] FLAG: --pod-cidr=""
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902136    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902142    2557 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902144    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902148    2557 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902151    2557 flags.go:64] FLAG: --port="10250"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902154    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902157    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07c194d29bfa64b1e"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902160    2557 flags.go:64] FLAG: --qos-reserved=""
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902163    2557 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902165    2557 flags.go:64] FLAG: --register-node="true"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902168    2557 flags.go:64] FLAG: --register-with-taints=""
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902172    2557 flags.go:64] FLAG: --registry-burst="10"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902175    2557 flags.go:64] FLAG: --registry-qps="5"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902178    2557 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902181    2557 flags.go:64] FLAG: --reserved-memory=""
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902184    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 20:58:38.909663 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902187    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902190    2557 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902194    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902197    2557 flags.go:64] FLAG: --runonce="false"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902200    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902203    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902206    2557 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902209    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902211    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902214    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902217    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902220    2557 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902223    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902227    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902230    2557 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902233    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902236    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902239    2557 flags.go:64] FLAG: --system-cgroups=""
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902242    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902247    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902249    2557 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902252    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902256    2557 flags.go:64] FLAG: --tls-min-version=""
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902259    2557 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902262    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 20:58:38.910335 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902265    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902268    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902271    2557 flags.go:64] FLAG: --v="2"
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902274    2557 flags.go:64] FLAG: --version="false"
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902278    2557 flags.go:64] FLAG: --vmodule=""
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902281    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.902284    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902400    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902404    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902408    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902411    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902414    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902417    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902421    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902424    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902427    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902429    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902432    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902435    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902437    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902441    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:38.910934 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902444    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902446    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902449    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902451    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902456    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902458    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902461    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902464    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902466    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902469    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902471    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902474    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902478    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902481    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902484    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902486    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902489    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902492    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902494    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902497    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:38.911469 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902500    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902502    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902505    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902508    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902511    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902513    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902516    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902519    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902521    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902524    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902526    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902530    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902533    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902536    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902539    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902541    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902544    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902547    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902549    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:38.911987 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902552    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902554    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902557    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902559    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902562    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902564    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902567    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902569    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902572    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902575    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902577    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902581    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902585    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902590    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902594    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902597    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902599    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902602    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902605    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902607    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:38.912468 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902610    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902612    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902614    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902617    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902621    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902623    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902626    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902628    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902631    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902633    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902636    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902640    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902644    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902646    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902649    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902651    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902654    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902656    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902659    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902662    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:38.912960 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902664    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902667    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902669    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902672    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902674    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.902677    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.903107    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.910568    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.910589    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.911277    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.911284    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911349    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911354    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911357    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:38.913523 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911361    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911364    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911367    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911369    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911372    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911375    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911378    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911381    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911383    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911393    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911396    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911399    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911403    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911407    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911410    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911414    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911418    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911421    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911424    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:38.913905 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911427    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911430    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911433    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911436    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911439    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911441    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911444    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911447    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911450    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911453    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911455    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911458    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911461    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911464    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911466    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911469    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911472    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911474    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911477    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911480    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:38.914394 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911490    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911493    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911496    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911505    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911508    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911511    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911513    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911516    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911519    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911521    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911524    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911526    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911529    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911531    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911534    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911537    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911539    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911542    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911545    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911548    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:38.914880 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911551    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911554    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911557    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911560    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911562    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911565    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911568    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911570    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911573    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911575    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911578    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911580    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911583    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911585    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911588    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911590    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911598    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911601    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911604    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911606    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:38.915371 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911609    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911611    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911614    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911617    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911619    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911622    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911624    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911627    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911630    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911632    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911636    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911639    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911642    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911645    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911647    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911649    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:38.915865 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911652    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.911657    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911801    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911809    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911812    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911815    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911818    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911821    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911823    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911826    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911829    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911832    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911834    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911842    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911845    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 20:58:38.916269 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911848    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911850    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911853    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911855    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911858    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911861    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911863    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911866    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911868    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911871    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911873    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911876    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911879    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911881    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911884    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911886    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911889    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911891    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911894    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911897    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 20:58:38.916656 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911899    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911902    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911904    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911907    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911909    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911912    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911914    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911916    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911919    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911921    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911924    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911932    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911935    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911937    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911940    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911942    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911945    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911947    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911950    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911952    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 20:58:38.917228 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911955    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911957    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911960    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911962    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911965    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911982    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911986    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911991    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911994    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.911997    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912000    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912003    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912005    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912008    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912010    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912013    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912015    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912018    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912020    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 20:58:38.917707 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912023    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912027    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912030    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912033    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912036    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912046    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912050    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912053    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912056    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912059    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912061    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912064    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912067    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912070    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912073    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912076    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912078    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912082    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912084    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 20:58:38.918192 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912087    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912090    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912098    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912101    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912104    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912106    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912109    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:38.912111    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.912117    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.912213    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.914544    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.915424    2557 server.go:1019] "Starting client certificate rotation"
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.915556    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 20:58:38.918655 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.915590    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 20:58:38.918990 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.917988    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 20:58:38.921215 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.921188    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 20:58:38.932275 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.932257    2557 log.go:25] "Validated CRI v1 runtime API"
Sep 10 20:58:38.935949 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.935930    2557 log.go:25] "Validated CRI v1 image API"
Sep 10 20:58:38.936719 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.936704    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 20:58:38.940136 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.940118    2557 fs.go:135] Filesystem UUIDs: map[06e25970-309b-419e-bd86-83f306304ab4:/dev/nvme0n1p3 5a1bae96-6e70-4611-83b6-1bb1841593be:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 10 20:58:38.940238 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.940136    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 20:58:38.946218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.946078    2557 manager.go:216] Machine: {Timestamp:2026-09-10 20:58:38.944766789 +0000 UTC m=+0.287885032 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3125611 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22210e2362a6e9d151e9b8045639d0 SystemUUID:ec22210e-2362-a6e9-d151-e9b8045639d0 BootID:b83a29dc-f977-4369-b5d1-69c271d9ce12 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fc:a4:c0:fd:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fc:a4:c0:fd:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:be:e3:c3:9f:73 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 20:58:38.946218 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.946207    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 20:58:38.946350 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.946285    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 20:58:38.949261 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.949237    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 20:58:38.949381 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.949263    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-211.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 20:58:38.950527 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.950517    2557 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 20:58:38.950560 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.950529    2557 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 20:58:38.950560 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.950537    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 20:58:38.951205 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.951195    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 20:58:38.952831 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.952822    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 20:58:38.957530 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.957519    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 20:58:38.957642 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.957633    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 20:58:38.957675 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.957661    2557 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 20:58:38.957675 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.957671    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 20:58:38.957725 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.957683    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 20:58:38.957725 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.957692    2557 kubelet.go:398] "Adding apiserver pod source"
Sep 10 20:58:38.957725 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.957701    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 20:58:38.958047 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.958035    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 20:58:38.958105 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.958069    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 20:58:38.959934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.959911    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 20:58:38.960106 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.960088    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 20:58:38.962922 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.962906    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 20:58:38.963024 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.962944    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 20:58:38.964742 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964725    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 20:58:38.964801 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964749    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 20:58:38.964801 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964757    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 20:58:38.964801 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964764    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 20:58:38.964801 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964774    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 20:58:38.964801 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964781    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 20:58:38.964801 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964797    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 20:58:38.964801 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964802    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 20:58:38.965018 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964811    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 20:58:38.965018 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964820    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 20:58:38.965018 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964838    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 20:58:38.965018 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.964843    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 20:58:38.965403 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.965394    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 20:58:38.965438 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.965405    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 20:58:38.968910 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.968899    2557 server.go:1268] "Started kubelet"
Sep 10 20:58:38.969034 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.969010    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 20:58:38.969093 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.969007    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 20:58:38.969093 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.969073    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 20:58:38.972472 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.972452    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 20:58:38.972569 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.972507    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-211.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 20:58:38.972898 ip-10-0-140-211 systemd[1]: Started Kubernetes Kubelet.
Sep 10 20:58:38.973212 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:38.973184    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-211.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 20:58:38.973212 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:38.973197    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 20:58:38.975208 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.975187    2557 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 20:58:38.977220 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:38.977082    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 20:58:38.980509 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:38.979633    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-211.ec2.internal.18d41140190e434c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-211.ec2.internal,UID:ip-10-0-140-211.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-211.ec2.internal,},FirstTimestamp:2026-09-10 20:58:38.968873804 +0000 UTC m=+0.311992045,LastTimestamp:2026-09-10 20:58:38.968873804 +0000 UTC m=+0.311992045,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-211.ec2.internal,}"
Sep 10 20:58:38.981284 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.981269    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 20:58:38.981854 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.981838    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 20:58:38.981955 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:38.981937    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:38.982349 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.982333    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 20:58:38.982429 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.982357    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 20:58:38.982429 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.982335    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 20:58:38.982429 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.982428    2557 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 20:58:38.982586 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.982435    2557 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 20:58:38.983334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.982895    2557 factory.go:55] Registering systemd factory
Sep 10 20:58:38.983334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.982929    2557 factory.go:223] Registration of the systemd container factory successfully
Sep 10 20:58:38.983334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.983154    2557 factory.go:153] Registering CRI-O factory
Sep 10 20:58:38.983334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.983167    2557 factory.go:223] Registration of the crio container factory successfully
Sep 10 20:58:38.983334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.983235    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 20:58:38.983334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.983258    2557 factory.go:103] Registering Raw factory
Sep 10 20:58:38.983334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.983271    2557 manager.go:1196] Started watching for new ooms in manager
Sep 10 20:58:38.983945 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.983753    2557 manager.go:349] Starting recovery of all containers
Sep 10 20:58:38.987965 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.987944    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b27vp"
Sep 10 20:58:38.989172 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.989150    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 20:58:38.992352 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.992337    2557 manager.go:354] Recovery completed
Sep 10 20:58:38.994862 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:38.994823    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 20:58:38.995078 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:38.995057    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-211.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 20:58:38.998021 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.998007    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:38.999736 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.999722    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:38.999808 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.999748    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:38.999808 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:38.999761    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:39.000418 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.000402    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:39.000418 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.000416    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 20:58:39.000561 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.000426    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 20:58:39.000561 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.000442    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 20:58:39.001159 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.001140    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:39.001221 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.001180    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:39.001221 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.001193    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:39.001917 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.001902    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b27vp"
Sep 10 20:58:39.003069 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.003053    2557 policy_none.go:49] "None policy: Start"
Sep 10 20:58:39.003174 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.003074    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 20:58:39.003174 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.003088    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 20:58:39.005091 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.005075    2557 policy_none.go:47] "Start"
Sep 10 20:58:39.005174 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.004961    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-211.ec2.internal.18d411401ae527d3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-211.ec2.internal,UID:ip-10-0-140-211.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-211.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-211.ec2.internal,},FirstTimestamp:2026-09-10 20:58:38.999734227 +0000 UTC m=+0.342852468,LastTimestamp:2026-09-10 20:58:38.999734227 +0000 UTC m=+0.342852468,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-211.ec2.internal,}"
Sep 10 20:58:39.030619 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.030508    2557 manager.go:337] "Starting Device Plugin manager"
Sep 10 20:58:39.030727 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.030650    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 20:58:39.030727 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.030662    2557 server.go:87] "Starting device plugin registration server"
Sep 10 20:58:39.030952 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.030937    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 20:58:39.031058 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.030952    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 20:58:39.031058 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.031045    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 20:58:39.031161 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.031121    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 20:58:39.031161 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.031129    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 20:58:39.031609 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.031588    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 20:58:39.031684 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.031636    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.053383 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.053364    2557 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2591 into the system container \"/system.slice\": <nil>"
Sep 10 20:58:39.131637 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.131619    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:39.132540 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.132524    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:39.132599 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.132554    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:39.132599 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.132567    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:39.132599 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.132590    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.142705 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.142691    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.142748 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.142713    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-211.ec2.internal\": node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.191978 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.191958    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.255174 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.255151    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 20:58:39.255268 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.255198    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 20:58:39.255268 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.255219    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 20:58:39.255342 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.255291    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 20:58:39.259657 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.259642    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:39.292804 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.292778    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.355956 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.355933    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal"]
Sep 10 20:58:39.356065 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.356053    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:39.359440 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.359423    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:39.359517 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.359452    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:39.359517 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.359462    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:39.361629 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.361617    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:39.361751 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.361736    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.361804 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.361764    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:39.362450 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.362437    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:39.362520 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.362462    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:39.362520 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.362480    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:39.362582 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.362459    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:39.362582 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.362547    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:39.362582 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.362557    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:39.364728 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.364710    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.364799 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.364750    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:39.365506 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.365488    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:39.365572 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.365523    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:39.365572 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.365540    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:39.385452 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.385428    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68fca22effbed069bcacb26330a72d8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal\" (UID: \"68fca22effbed069bcacb26330a72d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.385555 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.385462    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68fca22effbed069bcacb26330a72d8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal\" (UID: \"68fca22effbed069bcacb26330a72d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.385555 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.385482    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/42033061b2efc1a5de4a9db1dfbac058-config\") pod \"kube-apiserver-proxy-ip-10-0-140-211.ec2.internal\" (UID: \"42033061b2efc1a5de4a9db1dfbac058\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.391548 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.391531    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-211.ec2.internal\" not found" node="ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.393398 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.393380    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.396043 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.396025    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-211.ec2.internal\" not found" node="ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.486266 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.486218    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68fca22effbed069bcacb26330a72d8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal\" (UID: \"68fca22effbed069bcacb26330a72d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.486365 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.486291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68fca22effbed069bcacb26330a72d8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal\" (UID: \"68fca22effbed069bcacb26330a72d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.486365 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.486316    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68fca22effbed069bcacb26330a72d8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal\" (UID: \"68fca22effbed069bcacb26330a72d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.486365 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.486334    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68fca22effbed069bcacb26330a72d8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal\" (UID: \"68fca22effbed069bcacb26330a72d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.486365 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.486349    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/42033061b2efc1a5de4a9db1dfbac058-config\") pod \"kube-apiserver-proxy-ip-10-0-140-211.ec2.internal\" (UID: \"42033061b2efc1a5de4a9db1dfbac058\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.486501 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.486381    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/42033061b2efc1a5de4a9db1dfbac058-config\") pod \"kube-apiserver-proxy-ip-10-0-140-211.ec2.internal\" (UID: \"42033061b2efc1a5de4a9db1dfbac058\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.494174 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.494157    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.594773 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.594744    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.695214 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:39.695196    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:39.695263 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.695214    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:39.699302 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:39.699288    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:41.119444 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:41.119394    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:41.218478 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.218455    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:41.220373 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:41.220358    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:41.239532 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.239518    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 20:58:41.239633 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.239618    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 20:58:41.239694 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.239665    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 20:58:41.321438 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:41.321418    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:41.331420 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.331394    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 20:53:40 +0000 UTC" deadline="2028-06-16 11:02:25.977523336 +0000 UTC"
Sep 10 20:58:41.331420 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.331416    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15470h3m44.64610981s"
Sep 10 20:58:41.421971 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:41.421947    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:41.463049 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.463024    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:41.509621 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.509599    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 20:58:41.510769 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.510749    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 20:58:41.510860 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.510782    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 20:58:41.510860 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.510793    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeHasSufficientPID"
Sep 10 20:58:41.522329 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:41.522315    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:41.550853 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:41.550824    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68fca22effbed069bcacb26330a72d8f.slice/crio-0a808ddf7806276aee91e0c275abd1005571e187e624056d0ee5cb9565569d87 WatchSource:0}: Error finding container 0a808ddf7806276aee91e0c275abd1005571e187e624056d0ee5cb9565569d87: Status 404 returned error can't find the container with id 0a808ddf7806276aee91e0c275abd1005571e187e624056d0ee5cb9565569d87
Sep 10 20:58:41.555212 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.555189    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 20:58:41.582628 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.582592    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal" event={"ID":"68fca22effbed069bcacb26330a72d8f","Type":"ContainerStarted","Data":"0a808ddf7806276aee91e0c275abd1005571e187e624056d0ee5cb9565569d87"}
Sep 10 20:58:41.623034 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:41.623008    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:41.638361 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:41.638342    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42033061b2efc1a5de4a9db1dfbac058.slice/crio-6610f5437bf1b8f77e036d846a66525df6a6b16a9e2b8475ad1dc447254a0070 WatchSource:0}: Error finding container 6610f5437bf1b8f77e036d846a66525df6a6b16a9e2b8475ad1dc447254a0070: Status 404 returned error can't find the container with id 6610f5437bf1b8f77e036d846a66525df6a6b16a9e2b8475ad1dc447254a0070
Sep 10 20:58:41.723815 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:41.723767    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-211.ec2.internal\" not found"
Sep 10 20:58:41.781613 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.781591    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:41.806743 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.806721    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:41.823971 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.823953    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 20:58:41.824865 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.824850    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal"
Sep 10 20:58:41.837215 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:41.837202    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 20:58:42.208979 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.208894    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:42.282701 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.282662    2557 apiserver.go:52] "Watching apiserver"
Sep 10 20:58:42.290228 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.290203    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 20:58:42.292132 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.292102    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-bn477","openshift-multus/multus-additional-cni-plugins-nz8tw","openshift-multus/multus-knbc6","openshift-multus/network-metrics-daemon-lkmvb","openshift-network-diagnostics/network-check-target-r86bc","kube-system/global-pull-secret-syncer-6chtx","kube-system/konnectivity-agent-pjjhk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal","openshift-network-operator/iptables-alerter-d462d","openshift-ovn-kubernetes/ovnkube-node-65xvx","kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8sv6m","openshift-dns/node-resolver-z7b7f"]
Sep 10 20:58:42.294485 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.294460    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.297633 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.297393    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.299649 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.299620    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.300188 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.300166    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 20:58:42.300271 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.300189    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 20:58:42.300365 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.300268    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6hxkt\""
Sep 10 20:58:42.302147 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.302127    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:42.302236 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.302205    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:42.304392 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.304374    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:42.304480 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.304443    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:42.306139 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.306121    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 20:58:42.306592 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.306572    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.306704 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.306649    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:42.309188 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.309170    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.311564 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.311542    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.314117 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.314091    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.316472 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.316452    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.317183 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317162    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-etc-kubernetes\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.317349 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317330    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-sys\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.317518 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317503    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-tmp\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.317694 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317659    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-multus-certs\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.317808 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317712    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d19bacb-b122-4c14-8f38-99989bc66f71-kubelet-config\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.317808 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317737    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysctl-conf\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.317808 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317760    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80156b77-2422-4f5e-93fa-cc6a4878e15b-agent-certs\") pod \"konnectivity-agent-pjjhk\" (UID: \"80156b77-2422-4f5e-93fa-cc6a4878e15b\") " pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.317808 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317783    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-lib-modules\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.317808 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317802    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-hostroot\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317824    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-conf-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317850    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317881    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-systemd\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317909    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-run\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317933    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-tuned\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317956    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-cni-multus\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.317980    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n275g\" (UniqueName: \"kubernetes.io/projected/c934801f-9482-4a87-b9d7-a153a5be0e5e-kube-api-access-n275g\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:42.318052 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318021    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d19bacb-b122-4c14-8f38-99989bc66f71-dbus\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318055    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318083    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-kubernetes\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318121    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysctl-d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318157    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-socket-dir-parent\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318181    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-kubelet\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318203    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfk4q\" (UniqueName: \"kubernetes.io/projected/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-kube-api-access-zfk4q\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysconfig\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318247    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lc2d\" (UniqueName: \"kubernetes.io/projected/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-kube-api-access-5lc2d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318270    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-system-cni-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318298    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-cni-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318321    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-cnibin\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318343    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-netns\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-daemon-config\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.318423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318407    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:42.319092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318441    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-var-lib-kubelet\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.319092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318478    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-k8s-cni-cncf-io\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.319092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318516    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-cni-bin\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.319092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318542    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80156b77-2422-4f5e-93fa-cc6a4878e15b-konnectivity-ca\") pod \"konnectivity-agent-pjjhk\" (UID: \"80156b77-2422-4f5e-93fa-cc6a4878e15b\") " pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.319092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318580    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-modprobe-d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.319092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318608    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-host\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.319092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.318631    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-cni-binary-copy\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.319390 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.319371    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.320463 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.320444    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 20:58:42.320532 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.320517    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 20:58:42.320641 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.320625    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 20:58:42.320726 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.320675    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 20:58:42.321009 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.320987    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 20:58:42.321314 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321239    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 20:58:42.321404 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321359    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sxn8g\""
Sep 10 20:58:42.321467 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321456    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-225jt\""
Sep 10 20:58:42.321518 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321488    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 20:58:42.321627 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321569    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hj6np\""
Sep 10 20:58:42.321627 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321489    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 20:58:42.321627 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321595    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 20:58:42.321627 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.321619    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.322092    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.322120    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.322243    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.322556    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.322568    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cbvg6\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.323038    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.323084    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jwr4b\""
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.323418    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.324281 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.323588    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 20:58:42.325031 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.324300    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 20:58:42.325031 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.324421    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 20:58:42.325031 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.324864    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v479g\""
Sep 10 20:58:42.325779 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.325726    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 20:58:42.325995 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.325976    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jp6cz\""
Sep 10 20:58:42.326085 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.326056    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 20:58:42.326186 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.326166    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 20:58:42.326244 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.326219    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 20:58:42.331044 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.331020    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 20:58:42.331044 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.331027    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 20:58:42.331190 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.331177    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7znqc\""
Sep 10 20:58:42.331254 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.331243    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 20:58:42.335940 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.335921    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 20:58:42.348854 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.348835    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zj67j"
Sep 10 20:58:42.366823 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.366805    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zj67j"
Sep 10 20:58:42.407496 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.407470    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 20:58:42.419804 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419725    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-conf-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.419804 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419764    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:42.419804 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419797    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-conf-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419811    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-ovn\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419836    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-systemd\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419859    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n275g\" (UniqueName: \"kubernetes.io/projected/c934801f-9482-4a87-b9d7-a153a5be0e5e-kube-api-access-n275g\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419883    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d19bacb-b122-4c14-8f38-99989bc66f71-dbus\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419905    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419932    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419969    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovnkube-script-lib\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.420025 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.419994    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p24zp\" (UniqueName: \"kubernetes.io/projected/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-kube-api-access-p24zp\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-device-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420088    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-kubernetes\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420113    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-kubelet\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420137    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-system-cni-dir\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.420163    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420161    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-registration-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420219    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.420235    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret podName:5d19bacb-b122-4c14-8f38-99989bc66f71 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:42.920214075 +0000 UTC m=+2.939619392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret") pod "global-pull-secret-syncer-6chtx" (UID: "5d19bacb-b122-4c14-8f38-99989bc66f71") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420296    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-kubelet\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420298    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-systemd\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.420398 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420382    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d19bacb-b122-4c14-8f38-99989bc66f71-dbus\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420408    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-kubernetes\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420463    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncd87\" (UniqueName: \"kubernetes.io/projected/68a27caa-f787-4f09-94ed-c590cd4289d2-kube-api-access-ncd87\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420508    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-hosts-file\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420536    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lc2d\" (UniqueName: \"kubernetes.io/projected/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-kube-api-access-5lc2d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420561    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-system-cni-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420708    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-cni-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420718    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-system-cni-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420734    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-cnibin\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420772    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80156b77-2422-4f5e-93fa-cc6a4878e15b-konnectivity-ca\") pod \"konnectivity-agent-pjjhk\" (UID: \"80156b77-2422-4f5e-93fa-cc6a4878e15b\") " pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420789    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-cnibin\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420804    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-cni-dir\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420816    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npn2r\" (UniqueName: \"kubernetes.io/projected/77465bf6-105a-44bb-ad82-66504328ad5f-kube-api-access-npn2r\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420837    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420855    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420878    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-var-lib-kubelet\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420907    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-k8s-cni-cncf-io\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.420980 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420959    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-var-lib-kubelet\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420984    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.420993    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-k8s-cni-cncf-io\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421035    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-var-lib-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421064    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-node-log\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421090    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-log-socket\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-cni-binary-copy\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421138    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-cni-netd\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421165    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovnkube-config\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421186    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-sys-fs\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421211    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysctl-conf\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421232    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80156b77-2422-4f5e-93fa-cc6a4878e15b-agent-certs\") pod \"konnectivity-agent-pjjhk\" (UID: \"80156b77-2422-4f5e-93fa-cc6a4878e15b\") " pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421256    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-cnibin\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421280    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421303    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421325    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovn-node-metrics-cert\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421351    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-hostroot\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.421698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421355    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysctl-conf\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421355    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80156b77-2422-4f5e-93fa-cc6a4878e15b-konnectivity-ca\") pod \"konnectivity-agent-pjjhk\" (UID: \"80156b77-2422-4f5e-93fa-cc6a4878e15b\") " pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421420    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg6qd\" (UniqueName: \"kubernetes.io/projected/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-kube-api-access-lg6qd\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421429    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-hostroot\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421461    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-run\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421487    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-tuned\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-cni-multus\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421537    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-serviceca\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421583    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421609    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysctl-d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421631    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-socket-dir-parent\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421649    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-cni-binary-copy\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421654    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfk4q\" (UniqueName: \"kubernetes.io/projected/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-kube-api-access-zfk4q\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421677    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421724    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-kubelet\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421729    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-run\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421748    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-systemd-units\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.422325 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421773    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysconfig\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421775    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-cni-multus\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421795    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-netns\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421783    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-daemon-config\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421891    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-run-netns\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421909    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysctl-d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421921    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-cni-bin\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421945    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-host\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421968    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-sysconfig\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421968    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb4519e9-751b-444c-ad50-64e2fca69168-iptables-alerter-script\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422006    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-modprobe-d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422022    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-var-lib-cni-bin\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422030    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-host\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.422041    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422079    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-host\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.421968    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-socket-dir-parent\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423138 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422102    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-etc-kubernetes\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.422119    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs podName:c934801f-9482-4a87-b9d7-a153a5be0e5e nodeName:}" failed. No retries permitted until 2026-09-10 20:58:42.922103554 +0000 UTC m=+2.941508869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs") pod "network-metrics-daemon-lkmvb" (UID: "c934801f-9482-4a87-b9d7-a153a5be0e5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422058    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-etc-kubernetes\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422146    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-netns\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422154    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt4nx\" (UniqueName: \"kubernetes.io/projected/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-kube-api-access-mt4nx\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422176    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-modprobe-d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422196    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb4519e9-751b-444c-ad50-64e2fca69168-host-slash\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422224    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-slash\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422248    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-systemd\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422272    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-tmp-dir\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422298    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-sys\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422322    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-tmp\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422347    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-multus-certs\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422372    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d19bacb-b122-4c14-8f38-99989bc66f71-kubelet-config\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422395    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-etc-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422406    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-sys\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422418    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-cni-bin\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.423994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422451    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-host-run-multus-certs\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.424851 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422462    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-env-overrides\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.424851 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422488    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzkjz\" (UniqueName: \"kubernetes.io/projected/bb4519e9-751b-444c-ad50-64e2fca69168-kube-api-access-qzkjz\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.424851 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422511    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-socket-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.424851 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422512    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-multus-daemon-config\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.424851 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422517    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d19bacb-b122-4c14-8f38-99989bc66f71-kubelet-config\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.424851 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422536    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-lib-modules\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.424851 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.422754    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-lib-modules\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.425881 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.425856    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-etc-tuned\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.426196 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.426173    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80156b77-2422-4f5e-93fa-cc6a4878e15b-agent-certs\") pod \"konnectivity-agent-pjjhk\" (UID: \"80156b77-2422-4f5e-93fa-cc6a4878e15b\") " pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.426307 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.426266    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-tmp\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.434877 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.434784    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfk4q\" (UniqueName: \"kubernetes.io/projected/d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b-kube-api-access-zfk4q\") pod \"multus-knbc6\" (UID: \"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b\") " pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.435540 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.435506    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n275g\" (UniqueName: \"kubernetes.io/projected/c934801f-9482-4a87-b9d7-a153a5be0e5e-kube-api-access-n275g\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:42.442296 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.442177    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:42.442296 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.442203    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:42.442296 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.442215    2557 projected.go:196] Error preparing data for projected volume kube-api-access-b7vch for pod openshift-network-diagnostics/network-check-target-r86bc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:42.442296 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.442265    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch podName:0b49b59b-81ba-4c02-851c-3cfa4db12c79 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:42.942248271 +0000 UTC m=+2.961653588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b7vch" (UniqueName: "kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch") pod "network-check-target-r86bc" (UID: "0b49b59b-81ba-4c02-851c-3cfa4db12c79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:42.444517 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.444498    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lc2d\" (UniqueName: \"kubernetes.io/projected/82e9095b-fa99-4326-9a9e-49ea1ed6cee6-kube-api-access-5lc2d\") pod \"tuned-8sv6m\" (UID: \"82e9095b-fa99-4326-9a9e-49ea1ed6cee6\") " pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.523491 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523421    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-cnibin\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.523491 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523473    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.523709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523501    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.523709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523529    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-cnibin\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.523709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523573    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovn-node-metrics-cert\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.523709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523601    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg6qd\" (UniqueName: \"kubernetes.io/projected/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-kube-api-access-lg6qd\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.523709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523619    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.523709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523640    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-serviceca\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.523709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523665    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523715    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523730    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-kubelet\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523729    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523748    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-systemd-units\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523799    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-systemd-units\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523845    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-kubelet\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523918    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-run-netns\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523950    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-host\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.523975    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb4519e9-751b-444c-ad50-64e2fca69168-iptables-alerter-script\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524008    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt4nx\" (UniqueName: \"kubernetes.io/projected/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-kube-api-access-mt4nx\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524033    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb4519e9-751b-444c-ad50-64e2fca69168-host-slash\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524055    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-slash\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524077    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-systemd\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524104 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524099    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-tmp-dir\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524128    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-etc-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524146    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-serviceca\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524153    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-cni-bin\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524174    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-env-overrides\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzkjz\" (UniqueName: \"kubernetes.io/projected/bb4519e9-751b-444c-ad50-64e2fca69168-kube-api-access-qzkjz\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524211    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb4519e9-751b-444c-ad50-64e2fca69168-host-slash\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524234    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-socket-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524251    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-run-netns\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524278    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-ovn\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524283    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524294    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-host\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524324    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524341    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-slash\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524374    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524385    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-systemd\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524404    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovnkube-script-lib\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.524784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524429    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p24zp\" (UniqueName: \"kubernetes.io/projected/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-kube-api-access-p24zp\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524471    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-device-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524531    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-system-cni-dir\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524585    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-registration-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524632    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524645    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-tmp-dir\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524709    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncd87\" (UniqueName: \"kubernetes.io/projected/68a27caa-f787-4f09-94ed-c590cd4289d2-kube-api-access-ncd87\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524726    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-etc-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524735    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-hosts-file\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524771    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npn2r\" (UniqueName: \"kubernetes.io/projected/77465bf6-105a-44bb-ad82-66504328ad5f-kube-api-access-npn2r\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524828    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb4519e9-751b-444c-ad50-64e2fca69168-iptables-alerter-script\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524870    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524884    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-run-ovn\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524915    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524959    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524998    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-var-lib-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525023    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-node-log\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.525558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525057    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-socket-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-log-socket\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525102    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-cni-netd\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovnkube-config\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525123    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525141    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-sys-fs\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525205    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-env-overrides\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525252    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-sys-fs\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525253    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77465bf6-105a-44bb-ad82-66504328ad5f-system-cni-dir\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525307    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-hosts-file\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525315    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-device-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.524773    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-cni-bin\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525361    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525367    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-var-lib-openvswitch\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525409    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525425    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525472    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/68a27caa-f787-4f09-94ed-c590cd4289d2-registration-dir\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.526204 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525519    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-log-socket\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526814 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525559    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-node-log\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526814 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525634    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovnkube-script-lib\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526814 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525668    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-host-cni-netd\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526814 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525903    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77465bf6-105a-44bb-ad82-66504328ad5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.526814 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.525959    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovnkube-config\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.526814 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.526433    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-ovn-node-metrics-cert\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.535852 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.535822    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg6qd\" (UniqueName: \"kubernetes.io/projected/ddab8ca1-5c10-41d1-8b83-5cf0ab36138a-kube-api-access-lg6qd\") pod \"node-resolver-z7b7f\" (UID: \"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a\") " pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.538821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.538803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt4nx\" (UniqueName: \"kubernetes.io/projected/ff2e3ab3-8f3a-4d95-9734-6c33367725fb-kube-api-access-mt4nx\") pod \"node-ca-bn477\" (UID: \"ff2e3ab3-8f3a-4d95-9734-6c33367725fb\") " pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.542769 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.542750    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncd87\" (UniqueName: \"kubernetes.io/projected/68a27caa-f787-4f09-94ed-c590cd4289d2-kube-api-access-ncd87\") pod \"aws-ebs-csi-driver-node-74ksl\" (UID: \"68a27caa-f787-4f09-94ed-c590cd4289d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.542878 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.542786    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzkjz\" (UniqueName: \"kubernetes.io/projected/bb4519e9-751b-444c-ad50-64e2fca69168-kube-api-access-qzkjz\") pod \"iptables-alerter-d462d\" (UID: \"bb4519e9-751b-444c-ad50-64e2fca69168\") " pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.542938 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.542908    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p24zp\" (UniqueName: \"kubernetes.io/projected/8be13ce3-5ffd-4463-8aa2-4bc9b94440ad-kube-api-access-p24zp\") pod \"ovnkube-node-65xvx\" (UID: \"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.543068 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.543050    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npn2r\" (UniqueName: \"kubernetes.io/projected/77465bf6-105a-44bb-ad82-66504328ad5f-kube-api-access-npn2r\") pod \"multus-additional-cni-plugins-nz8tw\" (UID: \"77465bf6-105a-44bb-ad82-66504328ad5f\") " pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.584609 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.584586    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal" event={"ID":"42033061b2efc1a5de4a9db1dfbac058","Type":"ContainerStarted","Data":"6610f5437bf1b8f77e036d846a66525df6a6b16a9e2b8475ad1dc447254a0070"}
Sep 10 20:58:42.606976 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.606953    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:58:42.616234 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.616216    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8sv6m"
Sep 10 20:58:42.627002 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.626985    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-knbc6"
Sep 10 20:58:42.634667 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.634647    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z7b7f"
Sep 10 20:58:42.642879 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.642861    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bn477"
Sep 10 20:58:42.649446 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.649420    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nz8tw"
Sep 10 20:58:42.656916 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.656896    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d462d"
Sep 10 20:58:42.664797 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.664775    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:58:42.670319 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.670284    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl"
Sep 10 20:58:42.873619 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.873528    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 20:58:42.927985 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.927954    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:42.928149 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:42.928021    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:42.928149 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.928129    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:42.928149 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.928139    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:42.928308 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.928202    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret podName:5d19bacb-b122-4c14-8f38-99989bc66f71 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:43.928181186 +0000 UTC m=+3.947586482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret") pod "global-pull-secret-syncer-6chtx" (UID: "5d19bacb-b122-4c14-8f38-99989bc66f71") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:42.928308 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:42.928225    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs podName:c934801f-9482-4a87-b9d7-a153a5be0e5e nodeName:}" failed. No retries permitted until 2026-09-10 20:58:43.928213057 +0000 UTC m=+3.947618355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs") pod "network-metrics-daemon-lkmvb" (UID: "c934801f-9482-4a87-b9d7-a153a5be0e5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:43.028553 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.028518    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:43.028712 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.028671    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:43.028712 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.028702    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:43.028712 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.028712    2557 projected.go:196] Error preparing data for projected volume kube-api-access-b7vch for pod openshift-network-diagnostics/network-check-target-r86bc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:43.028816 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.028759    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch podName:0b49b59b-81ba-4c02-851c-3cfa4db12c79 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:44.028745563 +0000 UTC m=+4.048150859 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7vch" (UniqueName: "kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch") pod "network-check-target-r86bc" (UID: "0b49b59b-81ba-4c02-851c-3cfa4db12c79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:43.369635 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.369610    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 20:53:42 +0000 UTC" deadline="2028-04-11 06:11:15.354242023 +0000 UTC"
Sep 10 20:58:43.369635 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.369635    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13881h12m31.984609398s"
Sep 10 20:58:43.379728 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:43.379705    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80156b77_2422_4f5e_93fa_cc6a4878e15b.slice/crio-1e3754cced6db67a51bc1980f924e3457419faf5622616e1f494b8c1288d085d WatchSource:0}: Error finding container 1e3754cced6db67a51bc1980f924e3457419faf5622616e1f494b8c1288d085d: Status 404 returned error can't find the container with id 1e3754cced6db67a51bc1980f924e3457419faf5622616e1f494b8c1288d085d
Sep 10 20:58:43.381030 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:43.381004    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e9095b_fa99_4326_9a9e_49ea1ed6cee6.slice/crio-67e5c868686fd682c3a1c02f5882064f677292216e3463addf0b190731575c52 WatchSource:0}: Error finding container 67e5c868686fd682c3a1c02f5882064f677292216e3463addf0b190731575c52: Status 404 returned error can't find the container with id 67e5c868686fd682c3a1c02f5882064f677292216e3463addf0b190731575c52
Sep 10 20:58:43.402661 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:43.402639    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e935b5_bc99_46c5_8cf7_a02f9cbbb36b.slice/crio-383663d0f14740cdcf00d165a7a5fb0842307af526b694d07c6fec2c37310147 WatchSource:0}: Error finding container 383663d0f14740cdcf00d165a7a5fb0842307af526b694d07c6fec2c37310147: Status 404 returned error can't find the container with id 383663d0f14740cdcf00d165a7a5fb0842307af526b694d07c6fec2c37310147
Sep 10 20:58:43.404081 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:43.404057    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77465bf6_105a_44bb_ad82_66504328ad5f.slice/crio-cffd6d17a0fcbe5ed8906ef48b66080c9608afa5717c85ba68607657644264dd WatchSource:0}: Error finding container cffd6d17a0fcbe5ed8906ef48b66080c9608afa5717c85ba68607657644264dd: Status 404 returned error can't find the container with id cffd6d17a0fcbe5ed8906ef48b66080c9608afa5717c85ba68607657644264dd
Sep 10 20:58:43.404879 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:43.404859    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff2e3ab3_8f3a_4d95_9734_6c33367725fb.slice/crio-539df8d1a8077cc9b4c6c90c0f3a231c398738657131946eb7d01705d054b6e3 WatchSource:0}: Error finding container 539df8d1a8077cc9b4c6c90c0f3a231c398738657131946eb7d01705d054b6e3: Status 404 returned error can't find the container with id 539df8d1a8077cc9b4c6c90c0f3a231c398738657131946eb7d01705d054b6e3
Sep 10 20:58:43.406562 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:43.406478    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb4519e9_751b_444c_ad50_64e2fca69168.slice/crio-b5efc8288cea10dda063b43725594e17d87b98a8dc9a96c6b1735c581de577b3 WatchSource:0}: Error finding container b5efc8288cea10dda063b43725594e17d87b98a8dc9a96c6b1735c581de577b3: Status 404 returned error can't find the container with id b5efc8288cea10dda063b43725594e17d87b98a8dc9a96c6b1735c581de577b3
Sep 10 20:58:43.408442 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:58:43.408397    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddab8ca1_5c10_41d1_8b83_5cf0ab36138a.slice/crio-f0cff021ff63b617c28269a6282097e000235c403cc8a60e253faa45d2112fa5 WatchSource:0}: Error finding container f0cff021ff63b617c28269a6282097e000235c403cc8a60e253faa45d2112fa5: Status 404 returned error can't find the container with id f0cff021ff63b617c28269a6282097e000235c403cc8a60e253faa45d2112fa5
Sep 10 20:58:43.579739 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.579559    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:43.579846 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.579819    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:43.590836 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.590802    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-knbc6" event={"ID":"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b","Type":"ContainerStarted","Data":"383663d0f14740cdcf00d165a7a5fb0842307af526b694d07c6fec2c37310147"}
Sep 10 20:58:43.594008 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.593985    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal" event={"ID":"42033061b2efc1a5de4a9db1dfbac058","Type":"ContainerStarted","Data":"372fe66966f95c1cd1c3866400bccc6b7739b8b303c3de85a3e054f0528ba5df"}
Sep 10 20:58:43.595092 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.595070    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl" event={"ID":"68a27caa-f787-4f09-94ed-c590cd4289d2","Type":"ContainerStarted","Data":"31879912e811142d686eb737550faecad618707d8a6139109ec9f939fca63a4d"}
Sep 10 20:58:43.596228 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.596208    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"62f1cf409a751d84f46ece80883e3ed7332834db286325f2f555e7777dc6ef0c"}
Sep 10 20:58:43.597311 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.597289    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d462d" event={"ID":"bb4519e9-751b-444c-ad50-64e2fca69168","Type":"ContainerStarted","Data":"b5efc8288cea10dda063b43725594e17d87b98a8dc9a96c6b1735c581de577b3"}
Sep 10 20:58:43.600012 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.599988    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bn477" event={"ID":"ff2e3ab3-8f3a-4d95-9734-6c33367725fb","Type":"ContainerStarted","Data":"539df8d1a8077cc9b4c6c90c0f3a231c398738657131946eb7d01705d054b6e3"}
Sep 10 20:58:43.602403 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.602356    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerStarted","Data":"cffd6d17a0fcbe5ed8906ef48b66080c9608afa5717c85ba68607657644264dd"}
Sep 10 20:58:43.603396 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.603362    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8sv6m" event={"ID":"82e9095b-fa99-4326-9a9e-49ea1ed6cee6","Type":"ContainerStarted","Data":"67e5c868686fd682c3a1c02f5882064f677292216e3463addf0b190731575c52"}
Sep 10 20:58:43.604502 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.604482    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjjhk" event={"ID":"80156b77-2422-4f5e-93fa-cc6a4878e15b","Type":"ContainerStarted","Data":"1e3754cced6db67a51bc1980f924e3457419faf5622616e1f494b8c1288d085d"}
Sep 10 20:58:43.605409 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.605389    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z7b7f" event={"ID":"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a","Type":"ContainerStarted","Data":"f0cff021ff63b617c28269a6282097e000235c403cc8a60e253faa45d2112fa5"}
Sep 10 20:58:43.615741 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.615705    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-211.ec2.internal" podStartSLOduration=2.615671815 podStartE2EDuration="2.615671815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:43.615672766 +0000 UTC m=+3.635078084" watchObservedRunningTime="2026-09-10 20:58:43.615671815 +0000 UTC m=+3.635077146"
Sep 10 20:58:43.935623 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.935545    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:43.935785 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:43.935622    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:43.935785 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.935759    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:43.935908 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.935819    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret podName:5d19bacb-b122-4c14-8f38-99989bc66f71 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:45.935801547 +0000 UTC m=+5.955206849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret") pod "global-pull-secret-syncer-6chtx" (UID: "5d19bacb-b122-4c14-8f38-99989bc66f71") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:43.935908 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.935884    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:43.936018 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:43.935916    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs podName:c934801f-9482-4a87-b9d7-a153a5be0e5e nodeName:}" failed. No retries permitted until 2026-09-10 20:58:45.935904618 +0000 UTC m=+5.955309920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs") pod "network-metrics-daemon-lkmvb" (UID: "c934801f-9482-4a87-b9d7-a153a5be0e5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:44.036960 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:44.036920    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:44.037128 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:44.037104    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:44.037128 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:44.037127    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:44.037246 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:44.037140    2557 projected.go:196] Error preparing data for projected volume kube-api-access-b7vch for pod openshift-network-diagnostics/network-check-target-r86bc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:44.037246 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:44.037198    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch podName:0b49b59b-81ba-4c02-851c-3cfa4db12c79 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:46.037180571 +0000 UTC m=+6.056585870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7vch" (UniqueName: "kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch") pod "network-check-target-r86bc" (UID: "0b49b59b-81ba-4c02-851c-3cfa4db12c79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:44.374897 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:44.370972    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 20:53:42 +0000 UTC" deadline="2028-04-17 11:34:27.13917741 +0000 UTC"
Sep 10 20:58:44.374897 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:44.371067    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14030h35m42.768115564s"
Sep 10 20:58:44.581722 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:44.581693    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:44.581883 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:44.581824    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:44.582237 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:44.582218    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:44.582353 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:44.582340    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:44.626109 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:44.626036    2557 generic.go:356] "Generic (PLEG): container finished" podID="68fca22effbed069bcacb26330a72d8f" containerID="77bfadcf1e9537083a5600ba374d58f45aa9ba9882b12c57902b3a2da8a60ba6" exitCode=0
Sep 10 20:58:44.626794 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:44.626762    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal" event={"ID":"68fca22effbed069bcacb26330a72d8f","Type":"ContainerDied","Data":"77bfadcf1e9537083a5600ba374d58f45aa9ba9882b12c57902b3a2da8a60ba6"}
Sep 10 20:58:45.579173 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:45.579098    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:45.579874 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:45.579264    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:45.643724 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:45.643669    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal" event={"ID":"68fca22effbed069bcacb26330a72d8f","Type":"ContainerStarted","Data":"4eaba1f0e9f8291f00d20cf7333ad065add457ed58671deceed500add30a20ec"}
Sep 10 20:58:45.957423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:45.957339    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:45.957593 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:45.957422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:45.957593 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:45.957567    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:45.957732 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:45.957626    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret podName:5d19bacb-b122-4c14-8f38-99989bc66f71 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:49.957609229 +0000 UTC m=+9.977014528 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret") pod "global-pull-secret-syncer-6chtx" (UID: "5d19bacb-b122-4c14-8f38-99989bc66f71") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:45.958058 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:45.958036    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:45.958146 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:45.958091    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs podName:c934801f-9482-4a87-b9d7-a153a5be0e5e nodeName:}" failed. No retries permitted until 2026-09-10 20:58:49.958075936 +0000 UTC m=+9.977481238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs") pod "network-metrics-daemon-lkmvb" (UID: "c934801f-9482-4a87-b9d7-a153a5be0e5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:46.058419 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:46.058383    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:46.058637 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:46.058592    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:46.058637 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:46.058614    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:46.058637 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:46.058627    2557 projected.go:196] Error preparing data for projected volume kube-api-access-b7vch for pod openshift-network-diagnostics/network-check-target-r86bc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:46.058855 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:46.058701    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch podName:0b49b59b-81ba-4c02-851c-3cfa4db12c79 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:50.058669051 +0000 UTC m=+10.078074348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7vch" (UniqueName: "kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch") pod "network-check-target-r86bc" (UID: "0b49b59b-81ba-4c02-851c-3cfa4db12c79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:46.579944 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:46.579893    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:46.580438 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:46.580019    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:46.580481 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:46.580456    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:46.580635 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:46.580589    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:47.579414 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:47.579375    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:47.579576 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:47.579512    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:48.580183 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:48.580149    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:48.580618 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:48.580296    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:48.580618 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:48.580344    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:48.580618 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:48.580445    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:49.579820 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:49.579785    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:49.580009 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:49.579910    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:49.992344 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:49.992268    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:49.992344 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:49.992326    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:49.992822 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:49.992436    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:49.992822 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:49.992465    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:49.992822 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:49.992511    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs podName:c934801f-9482-4a87-b9d7-a153a5be0e5e nodeName:}" failed. No retries permitted until 2026-09-10 20:58:57.992492531 +0000 UTC m=+18.011897831 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs") pod "network-metrics-daemon-lkmvb" (UID: "c934801f-9482-4a87-b9d7-a153a5be0e5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:49.992822 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:49.992530    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret podName:5d19bacb-b122-4c14-8f38-99989bc66f71 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:57.992520674 +0000 UTC m=+18.011925974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret") pod "global-pull-secret-syncer-6chtx" (UID: "5d19bacb-b122-4c14-8f38-99989bc66f71") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:50.093708 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:50.093658    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:50.093922 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:50.093852    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:50.093922 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:50.093879    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:50.093922 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:50.093891    2557 projected.go:196] Error preparing data for projected volume kube-api-access-b7vch for pod openshift-network-diagnostics/network-check-target-r86bc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:50.094121 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:50.093961    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch podName:0b49b59b-81ba-4c02-851c-3cfa4db12c79 nodeName:}" failed. No retries permitted until 2026-09-10 20:58:58.093933782 +0000 UTC m=+18.113339082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7vch" (UniqueName: "kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch") pod "network-check-target-r86bc" (UID: "0b49b59b-81ba-4c02-851c-3cfa4db12c79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:50.581511 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:50.581096    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:50.581511 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:50.581218    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:50.581511 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:50.581328    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:50.581511 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:50.581436    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:51.580311 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:51.579642    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:51.580311 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:51.579791    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:52.580080 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:52.580048    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:52.580229 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:52.580173    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:52.580229 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:52.580206    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:52.580341 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:52.580279    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:53.579708 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:53.579662    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:53.579825 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:53.579772    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:54.579370 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:54.579332    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:54.579821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:54.579331    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:54.579821 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:54.579450    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:54.579821 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:54.579553    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:55.579255 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:55.579213    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:55.579459 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:55.579331    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:56.579306 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:56.579231    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:56.579476 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:56.579370    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:56.579476 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:56.579436    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:56.579889 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:56.579519    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:57.579947 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:57.579914    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:57.580427 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:57.580036    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:58:58.054043 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:58.053985    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:58.054237 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:58.054081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:58.054237 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.054169    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:58.054237 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.054213    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:58.054377 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.054251    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs podName:c934801f-9482-4a87-b9d7-a153a5be0e5e nodeName:}" failed. No retries permitted until 2026-09-10 20:59:14.054230268 +0000 UTC m=+34.073635567 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs") pod "network-metrics-daemon-lkmvb" (UID: "c934801f-9482-4a87-b9d7-a153a5be0e5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:58:58.054377 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.054273    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret podName:5d19bacb-b122-4c14-8f38-99989bc66f71 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:14.054262219 +0000 UTC m=+34.073667520 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret") pod "global-pull-secret-syncer-6chtx" (UID: "5d19bacb-b122-4c14-8f38-99989bc66f71") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:58:58.155221 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:58.155181    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:58.155374 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.155355    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:58:58.155456 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.155380    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:58:58.155456 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.155394    2557 projected.go:196] Error preparing data for projected volume kube-api-access-b7vch for pod openshift-network-diagnostics/network-check-target-r86bc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:58.155456 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.155453    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch podName:0b49b59b-81ba-4c02-851c-3cfa4db12c79 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:14.155438052 +0000 UTC m=+34.174843348 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7vch" (UniqueName: "kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch") pod "network-check-target-r86bc" (UID: "0b49b59b-81ba-4c02-851c-3cfa4db12c79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:58:58.580183 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:58.580149    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:58:58.580602 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.580300    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:58:58.580602 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:58.580364    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:58:58.580602 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:58.580471    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:58:59.579733 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:58:59.579523    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:58:59.579887 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:58:59.579823    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:00.580015 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:00.579986    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:00.580469 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:00.580078    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:00.580469 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:00.580177    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:00.580469 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:00.580286    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:01.579401 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.579241    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:01.579486 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:01.579466    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:01.677805 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.677772    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-knbc6" event={"ID":"d2e935b5-bc99-46c5-8cf7-a02f9cbbb36b","Type":"ContainerStarted","Data":"15ed7a01f55ca7cb7a24a1b8e886bb60d1ba77aa1f2a85ae4575a95652b9da41"}
Sep 10 20:59:01.679364 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.679343    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl" event={"ID":"68a27caa-f787-4f09-94ed-c590cd4289d2","Type":"ContainerStarted","Data":"63b327990bf645734d66eed2b4a6e2a4322d3841568690527014d2d9f6a56502"}
Sep 10 20:59:01.682099 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682079    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 20:59:01.682419 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682399    2557 generic.go:356] "Generic (PLEG): container finished" podID="8be13ce3-5ffd-4463-8aa2-4bc9b94440ad" containerID="78f578636dadcffe7d670c389ec97eb9e641a8cd8d1c13ceef2c98afe654863a" exitCode=1
Sep 10 20:59:01.682502 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682475    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"22e6d633abb2a8083e5b9eee6e3bf1ae41d8a05380caf9011ae839bf81574407"}
Sep 10 20:59:01.682548 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682535    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"37958d1f9f8c366d9c783d09197317d590e80ec4244b8a27d65449a8fdb4c58c"}
Sep 10 20:59:01.682594 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682576    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"7d2f25bdad5ad9a013a3b9c37dc595a2c832d7faa0762134907aca1a64a0175c"}
Sep 10 20:59:01.682726 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682612    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"9c286c565f0e71f1243de2ec62c4a5b412f05273b1a9d49613e99c62a28762c3"}
Sep 10 20:59:01.682726 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682645    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerDied","Data":"78f578636dadcffe7d670c389ec97eb9e641a8cd8d1c13ceef2c98afe654863a"}
Sep 10 20:59:01.682726 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.682698    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"faf1fb8d1b5155259e25c7fecf6a2cd4c698eaa58a2d0deaf99d3f37acdac312"}
Sep 10 20:59:01.683985 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.683964    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bn477" event={"ID":"ff2e3ab3-8f3a-4d95-9734-6c33367725fb","Type":"ContainerStarted","Data":"5911670c9a366d08c94365d454e9834c3442550d0c4af0432fa66f0bba7f625b"}
Sep 10 20:59:01.685468 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.685446    2557 generic.go:356] "Generic (PLEG): container finished" podID="77465bf6-105a-44bb-ad82-66504328ad5f" containerID="e48081ed9ad71f1ae8fec3657468b0af7faf52a30c16a4f8d76e442cb8f76837" exitCode=0
Sep 10 20:59:01.685543 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.685524    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerDied","Data":"e48081ed9ad71f1ae8fec3657468b0af7faf52a30c16a4f8d76e442cb8f76837"}
Sep 10 20:59:01.687080 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.687054    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8sv6m" event={"ID":"82e9095b-fa99-4326-9a9e-49ea1ed6cee6","Type":"ContainerStarted","Data":"468b1078167170eba04b4a70f9a52144c44a44420f0ecd002ea64249ec318451"}
Sep 10 20:59:01.688598 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.688581    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pjjhk" event={"ID":"80156b77-2422-4f5e-93fa-cc6a4878e15b","Type":"ContainerStarted","Data":"bcdac912b21d454c950804eca3699cac2e2cc3d53650e97cec2ab0caba29a8b9"}
Sep 10 20:59:01.690785 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.690759    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z7b7f" event={"ID":"ddab8ca1-5c10-41d1-8b83-5cf0ab36138a","Type":"ContainerStarted","Data":"8886f00044809cca2fd8d4576349c75d85a2da91100e251ee8b10804d8a6fc34"}
Sep 10 20:59:01.727397 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.727358    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-knbc6" podStartSLOduration=3.917067968 podStartE2EDuration="21.727347584s" totalImagesPullingTime="17.810279616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:01.726499757 +0000 UTC m=+21.745905075" watchObservedRunningTime="2026-09-10 20:59:01.727347584 +0000 UTC m=+21.746752902"
Sep 10 20:59:01.727616 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.727599    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-211.ec2.internal" podStartSLOduration=20.727594734 podStartE2EDuration="20.727594734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:58:45.669884019 +0000 UTC m=+5.689289335" watchObservedRunningTime="2026-09-10 20:59:01.727594734 +0000 UTC m=+21.747000052"
Sep 10 20:59:01.760940 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.760871    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z7b7f" podStartSLOduration=4.1865187 podStartE2EDuration="21.760859518s" totalImagesPullingTime="17.574340818s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:01.760757938 +0000 UTC m=+21.780163256" watchObservedRunningTime="2026-09-10 20:59:01.760859518 +0000 UTC m=+21.780264836"
Sep 10 20:59:01.835993 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.835942    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bn477" podStartSLOduration=4.255988919 podStartE2EDuration="21.835927484s" totalImagesPullingTime="17.579938565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:01.796801007 +0000 UTC m=+21.816206325" watchObservedRunningTime="2026-09-10 20:59:01.835927484 +0000 UTC m=+21.855332802"
Sep 10 20:59:01.874811 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.874771    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pjjhk" podStartSLOduration=4.286894277 podStartE2EDuration="21.874754536s" totalImagesPullingTime="17.587860259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:01.874437167 +0000 UTC m=+21.893842485" watchObservedRunningTime="2026-09-10 20:59:01.874754536 +0000 UTC m=+21.894159856"
Sep 10 20:59:01.907556 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:01.907514    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8sv6m" podStartSLOduration=4.319408314 podStartE2EDuration="21.907496103s" totalImagesPullingTime="17.588087789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:01.907328482 +0000 UTC m=+21.926733800" watchObservedRunningTime="2026-09-10 20:59:01.907496103 +0000 UTC m=+21.926901423"
Sep 10 20:59:02.194013 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.193986    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:59:02.194825 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.194797    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:59:02.213334 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.213313    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 20:59:02.364306 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.364156    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T20:59:02.213331687Z","UUID":"dae40c5f-7d52-451d-ab8e-e26d5da1819e","Handler":null,"Name":"","Endpoint":""}
Sep 10 20:59:02.365885 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.365860    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 20:59:02.366000 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.365891    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 20:59:02.579139 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.579112    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:02.579300 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.579154    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:02.579300 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:02.579245    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:02.579420 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:02.579375    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:02.695706 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.695591    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d462d" event={"ID":"bb4519e9-751b-444c-ad50-64e2fca69168","Type":"ContainerStarted","Data":"fbbb1bf236925e7311a16d993b0fbed56cddaa2d5b761a00bd83e051b13840dd"}
Sep 10 20:59:02.698765 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:02.698665    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl" event={"ID":"68a27caa-f787-4f09-94ed-c590cd4289d2","Type":"ContainerStarted","Data":"7a9e5dc599d76b3cfd4e7fe13c068a08b7bb576aafde035e290b921db8c63898"}
Sep 10 20:59:03.580343 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:03.580138    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:03.580546 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:03.580439    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:03.704566 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:03.704516    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl" event={"ID":"68a27caa-f787-4f09-94ed-c590cd4289d2","Type":"ContainerStarted","Data":"80eb2d13049c977bb420718a9c05d00c7e3e52af0325c2f3a0123f84cf058166"}
Sep 10 20:59:03.708161 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:03.708137    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 20:59:03.708582 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:03.708546    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"7b831499f33fc8934a90feecacea41ddfb657121cb6fbfca4ea75490380803f6"}
Sep 10 20:59:03.708676 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:03.708592    2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 20:59:03.733589 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:03.733541    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d462d" podStartSLOduration=6.115871613 podStartE2EDuration="23.73352286s" totalImagesPullingTime="17.617651247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:02.714026459 +0000 UTC m=+22.733431807" watchObservedRunningTime="2026-09-10 20:59:03.73352286 +0000 UTC m=+23.752928180"
Sep 10 20:59:04.579906 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:04.579875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:04.580130 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:04.580019    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:04.580130 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:04.580064    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:04.580198 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:04.580149    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:05.111768 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:05.111738    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:59:05.112259 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:05.111860    2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 20:59:05.112439 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:05.112419    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pjjhk"
Sep 10 20:59:05.145772 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:05.145713    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-74ksl" podStartSLOduration=5.714307845 podStartE2EDuration="25.145697748s" totalImagesPullingTime="19.431389903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:03.733501525 +0000 UTC m=+23.752906844" watchObservedRunningTime="2026-09-10 20:59:05.145697748 +0000 UTC m=+25.165103075"
Sep 10 20:59:05.579311 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:05.579279    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:05.579498 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:05.579391    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:06.579509 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.579343    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:06.579940 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.579346    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:06.579940 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:06.579587    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:06.579940 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:06.579711    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:06.718132 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.718106    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 20:59:06.718489 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.718463    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"93834508f8c46a1f0bf225d9a0949eddfbb5a324f2abbc3900541815e393e751"}
Sep 10 20:59:06.718930 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.718905    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:59:06.719043 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.718965    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:59:06.719043 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.718966    2557 scope.go:117] "RemoveContainer" containerID="78f578636dadcffe7d670c389ec97eb9e641a8cd8d1c13ceef2c98afe654863a"
Sep 10 20:59:06.720718 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.720673    2557 generic.go:356] "Generic (PLEG): container finished" podID="77465bf6-105a-44bb-ad82-66504328ad5f" containerID="50ec27069fa62b8bd64e9f36590490477995bb3e8c91894b5dedfd4beb18c0f5" exitCode=0
Sep 10 20:59:06.720791 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.720719    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerDied","Data":"50ec27069fa62b8bd64e9f36590490477995bb3e8c91894b5dedfd4beb18c0f5"}
Sep 10 20:59:06.734644 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.734624    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:59:06.734776 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:06.734763    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:59:07.579513 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.579489    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:07.579615 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:07.579575    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:07.727710 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.727673    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 20:59:07.728130 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.728103    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" event={"ID":"8be13ce3-5ffd-4463-8aa2-4bc9b94440ad","Type":"ContainerStarted","Data":"851857b5c90f76d30a80dff3ced69633a62649fa9365eeb2f8365b3b694ff487"}
Sep 10 20:59:07.728239 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.728198    2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 20:59:07.730718 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.730677    2557 generic.go:356] "Generic (PLEG): container finished" podID="77465bf6-105a-44bb-ad82-66504328ad5f" containerID="19612496d4e858cf8c9e5444a99f93df192e0c719d355df5c567437a5eb2ebac" exitCode=0
Sep 10 20:59:07.730813 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.730728    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerDied","Data":"19612496d4e858cf8c9e5444a99f93df192e0c719d355df5c567437a5eb2ebac"}
Sep 10 20:59:07.744574 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.744554    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6chtx"]
Sep 10 20:59:07.744658 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.744634    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:07.744763 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:07.744729    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:07.749315 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.749283    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r86bc"]
Sep 10 20:59:07.749404 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.749373    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:07.749481 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:07.749463    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:07.772199 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.772176    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lkmvb"]
Sep 10 20:59:07.772283 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.772271    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:07.772390 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:07.772373    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:07.821068 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:07.821023    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx" podStartSLOduration=10.19298338 podStartE2EDuration="27.821009907s" totalImagesPullingTime="17.628026527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:07.784861811 +0000 UTC m=+27.804267131" watchObservedRunningTime="2026-09-10 20:59:07.821009907 +0000 UTC m=+27.840415225"
Sep 10 20:59:08.735915 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:08.735883    2557 generic.go:356] "Generic (PLEG): container finished" podID="77465bf6-105a-44bb-ad82-66504328ad5f" containerID="a71d95fda2b43842156c5ff60e582c5035a03f713bc038befefb91b71d986a03" exitCode=0
Sep 10 20:59:08.736272 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:08.735956    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerDied","Data":"a71d95fda2b43842156c5ff60e582c5035a03f713bc038befefb91b71d986a03"}
Sep 10 20:59:08.736272 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:08.736061    2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 20:59:09.161930 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:09.161766    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:59:09.579321 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:09.579279    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:09.579483 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:09.579325    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:09.579483 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:09.579423    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:09.579483 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:09.579428    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:09.579642 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:09.579520    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:09.579710 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:09.579665    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:11.579263 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:11.579227    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:11.579737 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:11.579265    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:11.579737 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:11.579227    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:11.579737 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:11.579357    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:11.579737 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:11.579461    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:11.579737 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:11.579547    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:13.579404 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.579372    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:13.579404 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.579384    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:13.579873 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.579372    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:13.579873 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:13.579462    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6chtx" podUID="5d19bacb-b122-4c14-8f38-99989bc66f71"
Sep 10 20:59:13.579873 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:13.579543    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r86bc" podUID="0b49b59b-81ba-4c02-851c-3cfa4db12c79"
Sep 10 20:59:13.579873 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:13.579611    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lkmvb" podUID="c934801f-9482-4a87-b9d7-a153a5be0e5e"
Sep 10 20:59:13.746601 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.746578    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-211.ec2.internal" event="NodeReady"
Sep 10 20:59:13.746729 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.746676    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 20:59:13.758026 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.758005    2557 generic.go:356] "Generic (PLEG): container finished" podID="77465bf6-105a-44bb-ad82-66504328ad5f" containerID="afbc59a8201fc16827789e53f0f8e7d21d13df441bc2fd31479cc2396ede4635" exitCode=0
Sep 10 20:59:13.758111 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.758060    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerDied","Data":"afbc59a8201fc16827789e53f0f8e7d21d13df441bc2fd31479cc2396ede4635"}
Sep 10 20:59:13.900728 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.900705    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mt2bd"]
Sep 10 20:59:13.914850 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.914824    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r6rnh"]
Sep 10 20:59:13.914932 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.914835    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:13.921304 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.921282    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 20:59:13.923572 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.923557    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 20:59:13.923834 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.923815    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tkv7t\""
Sep 10 20:59:13.923967 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.923892    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 20:59:13.926985 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.926970    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:13.943539 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.943519    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mt2bd"]
Sep 10 20:59:13.946784 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.946769    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 20:59:13.947421 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.947401    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 20:59:13.947662 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.947649    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vl2hm\""
Sep 10 20:59:13.950087 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:13.950069    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r6rnh"]
Sep 10 20:59:14.072731 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072710    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f171cba7-5c57-4da6-a497-750d801b1ce2-config-volume\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.072854 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072735    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.072854 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072758    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:14.072854 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072805    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f171cba7-5c57-4da6-a497-750d801b1ce2-tmp-dir\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.072854 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072842    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s569\" (UniqueName: \"kubernetes.io/projected/a483da08-36c2-4bb6-b1c8-495b59ef3d61-kube-api-access-7s569\") pod \"ingress-canary-mt2bd\" (UID: \"a483da08-36c2-4bb6-b1c8-495b59ef3d61\") " pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:14.072994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072861    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djmfq\" (UniqueName: \"kubernetes.io/projected/f171cba7-5c57-4da6-a497-750d801b1ce2-kube-api-access-djmfq\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.072994 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.072873    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 20:59:14.072994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072901    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:14.072994 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.072932    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret podName:5d19bacb-b122-4c14-8f38-99989bc66f71 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:46.072914303 +0000 UTC m=+66.092319602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret") pod "global-pull-secret-syncer-6chtx" (UID: "5d19bacb-b122-4c14-8f38-99989bc66f71") : object "kube-system"/"original-pull-secret" not registered
Sep 10 20:59:14.072994 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.072944    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:59:14.072994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.072954    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a483da08-36c2-4bb6-b1c8-495b59ef3d61-cert\") pod \"ingress-canary-mt2bd\" (UID: \"a483da08-36c2-4bb6-b1c8-495b59ef3d61\") " pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:14.072994 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.072992    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs podName:c934801f-9482-4a87-b9d7-a153a5be0e5e nodeName:}" failed. No retries permitted until 2026-09-10 20:59:46.072980224 +0000 UTC m=+66.092385529 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs") pod "network-metrics-daemon-lkmvb" (UID: "c934801f-9482-4a87-b9d7-a153a5be0e5e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 20:59:14.173674 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.173622    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s569\" (UniqueName: \"kubernetes.io/projected/a483da08-36c2-4bb6-b1c8-495b59ef3d61-kube-api-access-7s569\") pod \"ingress-canary-mt2bd\" (UID: \"a483da08-36c2-4bb6-b1c8-495b59ef3d61\") " pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:14.173674 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.173649    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djmfq\" (UniqueName: \"kubernetes.io/projected/f171cba7-5c57-4da6-a497-750d801b1ce2-kube-api-access-djmfq\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.173889 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.173700    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a483da08-36c2-4bb6-b1c8-495b59ef3d61-cert\") pod \"ingress-canary-mt2bd\" (UID: \"a483da08-36c2-4bb6-b1c8-495b59ef3d61\") " pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:14.173889 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.173742    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:14.173889 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.173766    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f171cba7-5c57-4da6-a497-750d801b1ce2-config-volume\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.173889 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.173789    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.173889 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.173868    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:14.173889 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.173880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f171cba7-5c57-4da6-a497-750d801b1ce2-tmp-dir\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.174134 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.173922    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls podName:f171cba7-5c57-4da6-a497-750d801b1ce2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:14.673904956 +0000 UTC m=+34.693310256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls") pod "dns-default-r6rnh" (UID: "f171cba7-5c57-4da6-a497-750d801b1ce2") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:14.174134 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.173873    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 20:59:14.174134 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.173968    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 20:59:14.174134 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.173981    2557 projected.go:196] Error preparing data for projected volume kube-api-access-b7vch for pod openshift-network-diagnostics/network-check-target-r86bc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:59:14.174134 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.174027    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch podName:0b49b59b-81ba-4c02-851c-3cfa4db12c79 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:46.174013717 +0000 UTC m=+66.193419030 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b7vch" (UniqueName: "kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch") pod "network-check-target-r86bc" (UID: "0b49b59b-81ba-4c02-851c-3cfa4db12c79") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 20:59:14.174309 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.174184    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f171cba7-5c57-4da6-a497-750d801b1ce2-tmp-dir\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.174362 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.174344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f171cba7-5c57-4da6-a497-750d801b1ce2-config-volume\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.178463 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.178287    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a483da08-36c2-4bb6-b1c8-495b59ef3d61-cert\") pod \"ingress-canary-mt2bd\" (UID: \"a483da08-36c2-4bb6-b1c8-495b59ef3d61\") " pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:14.187348 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.187325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s569\" (UniqueName: \"kubernetes.io/projected/a483da08-36c2-4bb6-b1c8-495b59ef3d61-kube-api-access-7s569\") pod \"ingress-canary-mt2bd\" (UID: \"a483da08-36c2-4bb6-b1c8-495b59ef3d61\") " pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:14.193723 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.193703    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djmfq\" (UniqueName: \"kubernetes.io/projected/f171cba7-5c57-4da6-a497-750d801b1ce2-kube-api-access-djmfq\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.280004 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.279982    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mt2bd"
Sep 10 20:59:14.590994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.590972    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mt2bd"]
Sep 10 20:59:14.591980 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:14.591953    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda483da08_36c2_4bb6_b1c8_495b59ef3d61.slice/crio-322185bd0fb445ddc3a4fc2d6dd46bd8f5de1f5ea1666864563da7b4684da2fc WatchSource:0}: Error finding container 322185bd0fb445ddc3a4fc2d6dd46bd8f5de1f5ea1666864563da7b4684da2fc: Status 404 returned error can't find the container with id 322185bd0fb445ddc3a4fc2d6dd46bd8f5de1f5ea1666864563da7b4684da2fc
Sep 10 20:59:14.678368 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.678347    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:14.678487 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.678458    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:14.678547 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:14.678538    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls podName:f171cba7-5c57-4da6-a497-750d801b1ce2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:15.678524967 +0000 UTC m=+35.697930262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls") pod "dns-default-r6rnh" (UID: "f171cba7-5c57-4da6-a497-750d801b1ce2") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:14.763455 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.763424    2557 generic.go:356] "Generic (PLEG): container finished" podID="77465bf6-105a-44bb-ad82-66504328ad5f" containerID="35187f1dac7765cd156a96cfe524ea5b3c5333c663f40d9c41df8bcf5f8a0a26" exitCode=0
Sep 10 20:59:14.763648 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.763508    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerDied","Data":"35187f1dac7765cd156a96cfe524ea5b3c5333c663f40d9c41df8bcf5f8a0a26"}
Sep 10 20:59:14.764718 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.764668    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mt2bd" event={"ID":"a483da08-36c2-4bb6-b1c8-495b59ef3d61","Type":"ContainerStarted","Data":"322185bd0fb445ddc3a4fc2d6dd46bd8f5de1f5ea1666864563da7b4684da2fc"}
Sep 10 20:59:14.955988 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.955964    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"]
Sep 10 20:59:14.969231 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.969213    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:14.978955 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.978929    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 20:59:14.979029 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.978929    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 20:59:14.979126 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.979108    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4g4zv\""
Sep 10 20:59:14.979211 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.979196    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 20:59:14.981099 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.981080    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 20:59:14.992643 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:14.992585    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"]
Sep 10 20:59:15.044995 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.044972    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"]
Sep 10 20:59:15.061955 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.061933    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"
Sep 10 20:59:15.070423 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.070364    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"]
Sep 10 20:59:15.071225 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.070822    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 20:59:15.071225 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.071103    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 20:59:15.071642 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.071265    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-w5qr2\""
Sep 10 20:59:15.081881 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.081861    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa136b74-710f-492b-9b2b-b24fdc0029a6-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.081983 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.081891    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa136b74-710f-492b-9b2b-b24fdc0029a6-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.081983 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.081910    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lj84\" (UniqueName: \"kubernetes.io/projected/aa136b74-710f-492b-9b2b-b24fdc0029a6-kube-api-access-2lj84\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.144016 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.143954    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"]
Sep 10 20:59:15.161435 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.161410    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69f66476c-vtvls"]
Sep 10 20:59:15.161573 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.161554    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.177377 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.177354    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 20:59:15.177717 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.177675    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 20:59:15.177717 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.177707    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t568q\""
Sep 10 20:59:15.179308 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.179124    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 20:59:15.179558 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.179433    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 20:59:15.179707 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.179634    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.180420 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.180371    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"]
Sep 10 20:59:15.183902 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.183430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa136b74-710f-492b-9b2b-b24fdc0029a6-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.183902 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.183465    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa136b74-710f-492b-9b2b-b24fdc0029a6-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.183902 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.183492    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lj84\" (UniqueName: \"kubernetes.io/projected/aa136b74-710f-492b-9b2b-b24fdc0029a6-kube-api-access-2lj84\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.183902 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.183522    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8kxb\" (UniqueName: \"kubernetes.io/projected/fe52bdbe-85cc-4e06-a7f2-f57853f61d2d-kube-api-access-s8kxb\") pod \"network-check-source-6f8b768455-z77n9\" (UID: \"fe52bdbe-85cc-4e06-a7f2-f57853f61d2d\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"
Sep 10 20:59:15.184607 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.184274    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aa136b74-710f-492b-9b2b-b24fdc0029a6-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.185211 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.185134    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 20:59:15.186017 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.185344    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bbgkn\""
Sep 10 20:59:15.186017 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.185617    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 20:59:15.186191 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.186168    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69f66476c-vtvls"]
Sep 10 20:59:15.187215 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.187185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aa136b74-710f-492b-9b2b-b24fdc0029a6-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.190075 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.190055    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 20:59:15.197938 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.197776    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 20:59:15.214870 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.214824    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lj84\" (UniqueName: \"kubernetes.io/projected/aa136b74-710f-492b-9b2b-b24fdc0029a6-kube-api-access-2lj84\") pod \"kube-storage-version-migrator-operator-6c8f84f897-sjd8s\" (UID: \"aa136b74-710f-492b-9b2b-b24fdc0029a6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.280695 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.280659    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"
Sep 10 20:59:15.284068 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284037    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8kxb\" (UniqueName: \"kubernetes.io/projected/fe52bdbe-85cc-4e06-a7f2-f57853f61d2d-kube-api-access-s8kxb\") pod \"network-check-source-6f8b768455-z77n9\" (UID: \"fe52bdbe-85cc-4e06-a7f2-f57853f61d2d\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"
Sep 10 20:59:15.284181 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284084    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.284181 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284131    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-trusted-ca\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.284181 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284164    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppq5g\" (UniqueName: \"kubernetes.io/projected/0fdee397-ee70-4c4f-87e3-95289083c592-kube-api-access-ppq5g\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.284332 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284193    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-certificates\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.284332 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284274    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-image-registry-private-configuration\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.284332 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284309    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cgrl\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-kube-api-access-2cgrl\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.284497 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-installation-pull-secrets\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.284497 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284413    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.284497 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284435    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-bound-sa-token\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.284640 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284528    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0fdee397-ee70-4c4f-87e3-95289083c592-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.284640 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.284568    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-ca-trust-extracted\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.303561 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.303540    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8kxb\" (UniqueName: \"kubernetes.io/projected/fe52bdbe-85cc-4e06-a7f2-f57853f61d2d-kube-api-access-s8kxb\") pod \"network-check-source-6f8b768455-z77n9\" (UID: \"fe52bdbe-85cc-4e06-a7f2-f57853f61d2d\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"
Sep 10 20:59:15.372811 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.372786    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"
Sep 10 20:59:15.385101 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385067    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.385203 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385106    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-trusted-ca\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385203 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385130    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppq5g\" (UniqueName: \"kubernetes.io/projected/0fdee397-ee70-4c4f-87e3-95289083c592-kube-api-access-ppq5g\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.385203 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385157    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-certificates\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385315 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385212    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-image-registry-private-configuration\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385315 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.385238    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:15.385315 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.385303    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls podName:0fdee397-ee70-4c4f-87e3-95289083c592 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:15.885284715 +0000 UTC m=+35.904690015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-ghgxz" (UID: "0fdee397-ee70-4c4f-87e3-95289083c592") : secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:15.385440 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385238    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cgrl\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-kube-api-access-2cgrl\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385526 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385500    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-installation-pull-secrets\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385747 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385725    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385845 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385759    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-bound-sa-token\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385845 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385815    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-certificates\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.385845 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385834    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0fdee397-ee70-4c4f-87e3-95289083c592-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.386017 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.385864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-ca-trust-extracted\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.386017 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.385904    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:15.386017 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.385919    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f66476c-vtvls: secret "image-registry-tls" not found
Sep 10 20:59:15.386017 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.385978    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls podName:cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b nodeName:}" failed. No retries permitted until 2026-09-10 20:59:15.885960108 +0000 UTC m=+35.905365418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls") pod "image-registry-69f66476c-vtvls" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b") : secret "image-registry-tls" not found
Sep 10 20:59:15.386213 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.386076    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-trusted-ca\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.386213 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.386194    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-ca-trust-extracted\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.386676 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.386622    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0fdee397-ee70-4c4f-87e3-95289083c592-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.388122 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.388102    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-image-registry-private-configuration\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.388360 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.388345    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-installation-pull-secrets\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.408605 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.408381    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-bound-sa-token\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.416044 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.416018    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppq5g\" (UniqueName: \"kubernetes.io/projected/0fdee397-ee70-4c4f-87e3-95289083c592-kube-api-access-ppq5g\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.419063 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.419039    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cgrl\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-kube-api-access-2cgrl\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.579767 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.579736    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:15.579767 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.579758    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:15.580048 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.580004    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:15.583669 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.583627    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5qc4z\""
Sep 10 20:59:15.585434 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.584767    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 20:59:15.585434 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.584963    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 20:59:15.585434 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.585109    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-947bp\""
Sep 10 20:59:15.585434 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.585428    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s"]
Sep 10 20:59:15.589305 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:15.589257    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa136b74_710f_492b_9b2b_b24fdc0029a6.slice/crio-0f2141da78a32dfbb3e7049f078a8c1c4fbd65953b5642342b7a5230e21006b5 WatchSource:0}: Error finding container 0f2141da78a32dfbb3e7049f078a8c1c4fbd65953b5642342b7a5230e21006b5: Status 404 returned error can't find the container with id 0f2141da78a32dfbb3e7049f078a8c1c4fbd65953b5642342b7a5230e21006b5
Sep 10 20:59:15.675981 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.675919    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-z77n9"]
Sep 10 20:59:15.679518 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:15.679490    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe52bdbe_85cc_4e06_a7f2_f57853f61d2d.slice/crio-7b53ed4d16c18ec24d500ec670f48ec05f97307f0cceea6df3f784716bb9d265 WatchSource:0}: Error finding container 7b53ed4d16c18ec24d500ec670f48ec05f97307f0cceea6df3f784716bb9d265: Status 404 returned error can't find the container with id 7b53ed4d16c18ec24d500ec670f48ec05f97307f0cceea6df3f784716bb9d265
Sep 10 20:59:15.688323 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.688301    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:15.688484 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.688459    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:15.688578 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.688530    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls podName:f171cba7-5c57-4da6-a497-750d801b1ce2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:17.688509963 +0000 UTC m=+37.707915264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls") pod "dns-default-r6rnh" (UID: "f171cba7-5c57-4da6-a497-750d801b1ce2") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:15.773454 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.773419    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" event={"ID":"77465bf6-105a-44bb-ad82-66504328ad5f","Type":"ContainerStarted","Data":"2701ca02e588e2f0080a91346c98a2dddd0542fc666bce52610200873bfa3a61"}
Sep 10 20:59:15.774891 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.774843    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9" event={"ID":"fe52bdbe-85cc-4e06-a7f2-f57853f61d2d","Type":"ContainerStarted","Data":"7b53ed4d16c18ec24d500ec670f48ec05f97307f0cceea6df3f784716bb9d265"}
Sep 10 20:59:15.776364 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.776330    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s" event={"ID":"aa136b74-710f-492b-9b2b-b24fdc0029a6","Type":"ContainerStarted","Data":"0f2141da78a32dfbb3e7049f078a8c1c4fbd65953b5642342b7a5230e21006b5"}
Sep 10 20:59:15.820199 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.820141    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nz8tw" podStartSLOduration=8.647319533 podStartE2EDuration="35.820122499s" totalImagesPullingTime="27.172802966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:15.819508997 +0000 UTC m=+35.838914317" watchObservedRunningTime="2026-09-10 20:59:15.820122499 +0000 UTC m=+35.839527815"
Sep 10 20:59:15.889747 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.889715    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:15.889906 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:15.889813    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:15.889906 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.889890    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:15.890023 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.889914    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f66476c-vtvls: secret "image-registry-tls" not found
Sep 10 20:59:15.890023 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.889975    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:15.890226 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.889979    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls podName:cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b nodeName:}" failed. No retries permitted until 2026-09-10 20:59:16.889958286 +0000 UTC m=+36.909363589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls") pod "image-registry-69f66476c-vtvls" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b") : secret "image-registry-tls" not found
Sep 10 20:59:15.890226 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:15.890197    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls podName:0fdee397-ee70-4c4f-87e3-95289083c592 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:16.890176011 +0000 UTC m=+36.909581321 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-ghgxz" (UID: "0fdee397-ee70-4c4f-87e3-95289083c592") : secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:16.897471 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:16.897437    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:16.897983 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:16.897601    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:16.897983 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:16.897721    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls podName:0fdee397-ee70-4c4f-87e3-95289083c592 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:18.897678723 +0000 UTC m=+38.917084040 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-ghgxz" (UID: "0fdee397-ee70-4c4f-87e3-95289083c592") : secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:16.897983 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:16.897749    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:16.897983 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:16.897919    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:16.897983 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:16.897942    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f66476c-vtvls: secret "image-registry-tls" not found
Sep 10 20:59:16.898218 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:16.897996    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls podName:cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b nodeName:}" failed. No retries permitted until 2026-09-10 20:59:18.897983284 +0000 UTC m=+38.917388581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls") pod "image-registry-69f66476c-vtvls" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b") : secret "image-registry-tls" not found
Sep 10 20:59:17.705463 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:17.705384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:17.705613 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:17.705504    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:17.705676 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:17.705613    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls podName:f171cba7-5c57-4da6-a497-750d801b1ce2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:21.705592554 +0000 UTC m=+41.724997854 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls") pod "dns-default-r6rnh" (UID: "f171cba7-5c57-4da6-a497-750d801b1ce2") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:17.784778 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:17.784746    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mt2bd" event={"ID":"a483da08-36c2-4bb6-b1c8-495b59ef3d61","Type":"ContainerStarted","Data":"ed5b9330d5d0637a4c80ce969d1353ffca40d27aa7293d8105905e8133d3327e"}
Sep 10 20:59:18.915211 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:18.915169    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:18.915695 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:18.915263    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:18.915695 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:18.915351    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:18.915695 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:18.915385    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f66476c-vtvls: secret "image-registry-tls" not found
Sep 10 20:59:18.915695 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:18.915359    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:18.915695 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:18.915470    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls podName:cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b nodeName:}" failed. No retries permitted until 2026-09-10 20:59:22.915429523 +0000 UTC m=+42.934834839 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls") pod "image-registry-69f66476c-vtvls" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b") : secret "image-registry-tls" not found
Sep 10 20:59:18.915695 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:18.915515    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls podName:0fdee397-ee70-4c4f-87e3-95289083c592 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:22.915498644 +0000 UTC m=+42.934903941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-ghgxz" (UID: "0fdee397-ee70-4c4f-87e3-95289083c592") : secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:20.792927 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:20.792892    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s" event={"ID":"aa136b74-710f-492b-9b2b-b24fdc0029a6","Type":"ContainerStarted","Data":"1d134020235318a336a19bc06201cc8831ae6190d313a44620ba4e5869d001c2"}
Sep 10 20:59:20.794465 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:20.794437    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9" event={"ID":"fe52bdbe-85cc-4e06-a7f2-f57853f61d2d","Type":"ContainerStarted","Data":"84b2343e37967d902ec728ba1c639dad80e71ccda8cf8a044b5fb3103e594078"}
Sep 10 20:59:20.825439 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:20.825387    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s" podStartSLOduration=2.2641471490000002 podStartE2EDuration="6.825371064s" totalImagesPullingTime="4.561223915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:20.824150115 +0000 UTC m=+40.843555429" watchObservedRunningTime="2026-09-10 20:59:20.825371064 +0000 UTC m=+40.844776383"
Sep 10 20:59:20.825560 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:20.825499    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mt2bd" podStartSLOduration=5.506859521 podStartE2EDuration="7.825491701s" totalImagesPullingTime="2.31863218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:17.812373756 +0000 UTC m=+37.831779074" watchObservedRunningTime="2026-09-10 20:59:20.825491701 +0000 UTC m=+40.844897020"
Sep 10 20:59:20.842659 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:20.842614    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-z77n9" podStartSLOduration=1.361761577 podStartE2EDuration="5.842597087s" totalImagesPullingTime="4.48083551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:20.842015466 +0000 UTC m=+40.861420784" watchObservedRunningTime="2026-09-10 20:59:20.842597087 +0000 UTC m=+40.862002407"
Sep 10 20:59:21.263873 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.263838    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cfvm7"]
Sep 10 20:59:21.300084 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.300062    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cfvm7"]
Sep 10 20:59:21.300216 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.300156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.305085 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.305065    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 20:59:21.305713 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.305669    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 20:59:21.305817 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.305785    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7vxwx\""
Sep 10 20:59:21.305817 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.305791    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 20:59:21.306183 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.306163    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 20:59:21.434703 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.434667    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdtrn\" (UniqueName: \"kubernetes.io/projected/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-api-access-cdtrn\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.434821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.434748    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-crio-socket\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.434821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.434776    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-data-volume\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.434911 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.434822    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.434911 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.434881    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.535627 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.535549    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-crio-socket\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.535627 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.535596    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-data-volume\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.535833 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.535648    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.535833 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.535709    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.535833 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.535793    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdtrn\" (UniqueName: \"kubernetes.io/projected/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-api-access-cdtrn\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.535833 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.535802    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-crio-socket\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.535973 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:21.535845    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:21.535973 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:21.535884    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:21.535973 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:21.535920    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:22.035898301 +0000 UTC m=+42.055303611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:21.535973 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:21.535942    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:22.035929791 +0000 UTC m=+42.055335107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:21.535973 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.535966    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-data-volume\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.546366 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.546342    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdtrn\" (UniqueName: \"kubernetes.io/projected/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-api-access-cdtrn\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:21.738501 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:21.738453    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:21.738707 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:21.738605    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:21.738707 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:21.738670    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls podName:f171cba7-5c57-4da6-a497-750d801b1ce2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:29.738655725 +0000 UTC m=+49.758061021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls") pod "dns-default-r6rnh" (UID: "f171cba7-5c57-4da6-a497-750d801b1ce2") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:22.042097 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.042055    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:22.042507 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.042163    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:22.042507 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.042214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:22.042507 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.042228    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:23.042212991 +0000 UTC m=+43.061618287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:22.042668 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.042526    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:22.042668 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.042606    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:23.042575993 +0000 UTC m=+43.061981306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:22.699116 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.699085    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"]
Sep 10 20:59:22.715285 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.715258    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"
Sep 10 20:59:22.717877 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.717849    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"]
Sep 10 20:59:22.718327 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.718307    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 20:59:22.719100 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.719076    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 20:59:22.719449 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.719427    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-4j7ph\""
Sep 10 20:59:22.849693 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.849664    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6twj\" (UniqueName: \"kubernetes.io/projected/c5aec7f5-66d7-4f47-980b-981e395d58a8-kube-api-access-t6twj\") pod \"migrator-8b4d697c5-f9d28\" (UID: \"c5aec7f5-66d7-4f47-980b-981e395d58a8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"
Sep 10 20:59:22.950992 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.950938    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6twj\" (UniqueName: \"kubernetes.io/projected/c5aec7f5-66d7-4f47-980b-981e395d58a8-kube-api-access-t6twj\") pod \"migrator-8b4d697c5-f9d28\" (UID: \"c5aec7f5-66d7-4f47-980b-981e395d58a8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"
Sep 10 20:59:22.950992 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.950976    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:22.951118 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.951029    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:22.951153 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.951134    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:22.951153 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.951141    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:22.951221 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.951160    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f66476c-vtvls: secret "image-registry-tls" not found
Sep 10 20:59:22.951221 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.951186    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls podName:0fdee397-ee70-4c4f-87e3-95289083c592 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:30.951167385 +0000 UTC m=+50.970572681 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-ghgxz" (UID: "0fdee397-ee70-4c4f-87e3-95289083c592") : secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:22.951221 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:22.951214    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls podName:cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b nodeName:}" failed. No retries permitted until 2026-09-10 20:59:30.95119823 +0000 UTC m=+50.970603543 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls") pod "image-registry-69f66476c-vtvls" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b") : secret "image-registry-tls" not found
Sep 10 20:59:22.960931 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:22.960903    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6twj\" (UniqueName: \"kubernetes.io/projected/c5aec7f5-66d7-4f47-980b-981e395d58a8-kube-api-access-t6twj\") pod \"migrator-8b4d697c5-f9d28\" (UID: \"c5aec7f5-66d7-4f47-980b-981e395d58a8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"
Sep 10 20:59:23.025193 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.025171    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"
Sep 10 20:59:23.051537 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.051516    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:23.051862 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.051551    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:23.051862 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:23.051674    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:23.051862 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:23.051716    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:23.051862 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:23.051732    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:25.051719575 +0000 UTC m=+45.071124870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:23.051862 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:23.051813    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:25.051792338 +0000 UTC m=+45.071197634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:23.353604 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.353573    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28"]
Sep 10 20:59:23.356621 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:23.356591    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5aec7f5_66d7_4f47_980b_981e395d58a8.slice/crio-104dc1d83de853eeae919219534bb6a8f0f9b3dc5142c6dc3b256f30a4a59417 WatchSource:0}: Error finding container 104dc1d83de853eeae919219534bb6a8f0f9b3dc5142c6dc3b256f30a4a59417: Status 404 returned error can't find the container with id 104dc1d83de853eeae919219534bb6a8f0f9b3dc5142c6dc3b256f30a4a59417
Sep 10 20:59:23.376009 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.375986    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z7b7f_ddab8ca1-5c10-41d1-8b83-5cf0ab36138a/dns-node-resolver/0.log"
Sep 10 20:59:23.391726 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.391703    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-6hgsr"]
Sep 10 20:59:23.422650 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.422626    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-6hgsr"]
Sep 10 20:59:23.422746 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.422727    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.425907 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.425888    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 20:59:23.425994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.425927    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 20:59:23.425994 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.425960    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 20:59:23.426937 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.426918    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-j47g4\""
Sep 10 20:59:23.427041 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.427029    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 20:59:23.555203 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.555169    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-signing-key\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.555342 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.555238    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-signing-cabundle\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.555342 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.555320    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s99sd\" (UniqueName: \"kubernetes.io/projected/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-kube-api-access-s99sd\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.656028 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.655959    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-signing-key\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.656141 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.656051    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-signing-cabundle\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.656141 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.656083    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s99sd\" (UniqueName: \"kubernetes.io/projected/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-kube-api-access-s99sd\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.656855 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.656835    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-signing-cabundle\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.658297 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.658281    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-signing-key\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.678597 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.678579    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s99sd\" (UniqueName: \"kubernetes.io/projected/29ee5818-9e7e-476d-b0b7-e112e4cdba5d-kube-api-access-s99sd\") pod \"service-ca-568869fdc5-6hgsr\" (UID: \"29ee5818-9e7e-476d-b0b7-e112e4cdba5d\") " pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.731811 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.731791    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-6hgsr"
Sep 10 20:59:23.806451 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:23.806414    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28" event={"ID":"c5aec7f5-66d7-4f47-980b-981e395d58a8","Type":"ContainerStarted","Data":"104dc1d83de853eeae919219534bb6a8f0f9b3dc5142c6dc3b256f30a4a59417"}
Sep 10 20:59:24.040709 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:24.040668    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-6hgsr"]
Sep 10 20:59:24.043954 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:24.043924    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ee5818_9e7e_476d_b0b7_e112e4cdba5d.slice/crio-5e580ce5befc637196a8a96bbd793b7112fd901d5a7d1c71510f9d4d67328baf WatchSource:0}: Error finding container 5e580ce5befc637196a8a96bbd793b7112fd901d5a7d1c71510f9d4d67328baf: Status 404 returned error can't find the container with id 5e580ce5befc637196a8a96bbd793b7112fd901d5a7d1c71510f9d4d67328baf
Sep 10 20:59:24.176566 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:24.176541    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bn477_ff2e3ab3-8f3a-4d95-9734-6c33367725fb/node-ca/0.log"
Sep 10 20:59:24.775600 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:24.775570    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mt2bd_a483da08-36c2-4bb6-b1c8-495b59ef3d61/serve-healthcheck-canary/0.log"
Sep 10 20:59:24.810963 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:24.810929    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-6hgsr" event={"ID":"29ee5818-9e7e-476d-b0b7-e112e4cdba5d","Type":"ContainerStarted","Data":"5e580ce5befc637196a8a96bbd793b7112fd901d5a7d1c71510f9d4d67328baf"}
Sep 10 20:59:25.067966 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:25.067930    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:25.068143 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:25.067988    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:25.068143 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:25.068073    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:25.068143 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:25.068138    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:29.068118287 +0000 UTC m=+49.087523582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:25.068323 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:25.068077    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:25.068323 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:25.068226    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:29.068208629 +0000 UTC m=+49.087613930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:25.816438 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:25.816403    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28" event={"ID":"c5aec7f5-66d7-4f47-980b-981e395d58a8","Type":"ContainerStarted","Data":"2a2d52af0e7dec98154d3b53cbd9265eff459a5043565c4f94b085284cff0a0d"}
Sep 10 20:59:25.816860 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:25.816496    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28" event={"ID":"c5aec7f5-66d7-4f47-980b-981e395d58a8","Type":"ContainerStarted","Data":"dd29f61e41e7a24ed58bcd2616eb54ae095078e738ca8b2785c7552af8d30226"}
Sep 10 20:59:25.853270 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:25.853222    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-f9d28" podStartSLOduration=2.167075409 podStartE2EDuration="3.853206803s" totalImagesPullingTime="1.686131394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:25.852706323 +0000 UTC m=+45.872111645" watchObservedRunningTime="2026-09-10 20:59:25.853206803 +0000 UTC m=+45.872612121"
Sep 10 20:59:26.821700 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:26.821645    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-6hgsr" event={"ID":"29ee5818-9e7e-476d-b0b7-e112e4cdba5d","Type":"ContainerStarted","Data":"c93a594d942dfd378df0c1ec860ae030992512c862893496717890b0f7593757"}
Sep 10 20:59:26.843510 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:26.843458    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-6hgsr" podStartSLOduration=1.23400375 podStartE2EDuration="3.843444604s" totalImagesPullingTime="2.609440854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:26.84288125 +0000 UTC m=+46.862286568" watchObservedRunningTime="2026-09-10 20:59:26.843444604 +0000 UTC m=+46.862849922"
Sep 10 20:59:29.097547 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:29.097507    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:29.098040 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:29.097562    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:29.098040 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:29.097724    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 20:59:29.098040 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:29.097805    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:37.097783741 +0000 UTC m=+57.117189043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : configmap "kube-rbac-proxy" not found
Sep 10 20:59:29.098225 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:29.098209    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:29.098279 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:29.098265    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:37.098250338 +0000 UTC m=+57.117655640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:29.802925 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:29.802892    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:29.803104 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:29.803076    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 20:59:29.803168 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:29.803155    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls podName:f171cba7-5c57-4da6-a497-750d801b1ce2 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:45.803135672 +0000 UTC m=+65.822540974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls") pod "dns-default-r6rnh" (UID: "f171cba7-5c57-4da6-a497-750d801b1ce2") : secret "dns-default-metrics-tls" not found
Sep 10 20:59:31.012968 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:31.012937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:31.013300 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:31.013029    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:31.013300 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:31.013097    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 20:59:31.013300 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:31.013114    2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:31.013300 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:31.013120    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f66476c-vtvls: secret "image-registry-tls" not found
Sep 10 20:59:31.013300 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:31.013160    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls podName:0fdee397-ee70-4c4f-87e3-95289083c592 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:47.013148654 +0000 UTC m=+67.032553950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-ghgxz" (UID: "0fdee397-ee70-4c4f-87e3-95289083c592") : secret "cluster-monitoring-operator-tls" not found
Sep 10 20:59:31.013300 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:31.013172    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls podName:cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b nodeName:}" failed. No retries permitted until 2026-09-10 20:59:47.01316649 +0000 UTC m=+67.032571786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls") pod "image-registry-69f66476c-vtvls" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b") : secret "image-registry-tls" not found
Sep 10 20:59:37.163183 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:37.163146    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:37.163183 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:37.163195    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:37.163792 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:37.163320    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:37.163792 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:37.163403    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls podName:04bdb162-6a86-43fc-a5b7-f3de85e4b1e5 nodeName:}" failed. No retries permitted until 2026-09-10 20:59:53.163382024 +0000 UTC m=+73.182787326 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cfvm7" (UID: "04bdb162-6a86-43fc-a5b7-f3de85e4b1e5") : secret "insights-runtime-extractor-tls" not found
Sep 10 20:59:37.163792 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:37.163640    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:39.749287 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:39.749259    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-65xvx"
Sep 10 20:59:45.826211 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:45.826167    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:45.828353 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:45.828333    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f171cba7-5c57-4da6-a497-750d801b1ce2-metrics-tls\") pod \"dns-default-r6rnh\" (UID: \"f171cba7-5c57-4da6-a497-750d801b1ce2\") " pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:46.089064 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.088997    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vl2hm\""
Sep 10 20:59:46.095368 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.095350    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:46.127394 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.127368    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:46.127520 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.127449    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:46.131836 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.131808    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 20:59:46.132268 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.132252    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 20:59:46.141082 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.141056    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c934801f-9482-4a87-b9d7-a153a5be0e5e-metrics-certs\") pod \"network-metrics-daemon-lkmvb\" (UID: \"c934801f-9482-4a87-b9d7-a153a5be0e5e\") " pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:46.141370 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.141344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d19bacb-b122-4c14-8f38-99989bc66f71-original-pull-secret\") pod \"global-pull-secret-syncer-6chtx\" (UID: \"5d19bacb-b122-4c14-8f38-99989bc66f71\") " pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:46.198243 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.198213    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5qc4z\""
Sep 10 20:59:46.203527 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.203509    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6chtx"
Sep 10 20:59:46.204567 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.204551    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lkmvb"
Sep 10 20:59:46.228105 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.228071    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:46.230409 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.230386    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7vch\" (UniqueName: \"kubernetes.io/projected/0b49b59b-81ba-4c02-851c-3cfa4db12c79-kube-api-access-b7vch\") pod \"network-check-target-r86bc\" (UID: \"0b49b59b-81ba-4c02-851c-3cfa4db12c79\") " pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:46.393634 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.393560    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r6rnh"]
Sep 10 20:59:46.396769 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:46.396739    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf171cba7_5c57_4da6_a497_750d801b1ce2.slice/crio-7a81d07079bf736d6322ccb675601ca0a8290525dd2e3cf54e6ecb0a61fa1329 WatchSource:0}: Error finding container 7a81d07079bf736d6322ccb675601ca0a8290525dd2e3cf54e6ecb0a61fa1329: Status 404 returned error can't find the container with id 7a81d07079bf736d6322ccb675601ca0a8290525dd2e3cf54e6ecb0a61fa1329
Sep 10 20:59:46.494651 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.494620    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6chtx"]
Sep 10 20:59:46.495030 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:46.494995    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d19bacb_b122_4c14_8f38_99989bc66f71.slice/crio-4369727f4da7c7cfc5e79cd7472b4aed8d68bbf445d0070d4401df3ad2b85953 WatchSource:0}: Error finding container 4369727f4da7c7cfc5e79cd7472b4aed8d68bbf445d0070d4401df3ad2b85953: Status 404 returned error can't find the container with id 4369727f4da7c7cfc5e79cd7472b4aed8d68bbf445d0070d4401df3ad2b85953
Sep 10 20:59:46.495508 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.495472    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lkmvb"]
Sep 10 20:59:46.495648 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:46.495610    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc934801f_9482_4a87_b9d7_a153a5be0e5e.slice/crio-42ed7ca7d8223d75fa4950aea20ed997aaaefd1765223ec81816ac56b8f202ac WatchSource:0}: Error finding container 42ed7ca7d8223d75fa4950aea20ed997aaaefd1765223ec81816ac56b8f202ac: Status 404 returned error can't find the container with id 42ed7ca7d8223d75fa4950aea20ed997aaaefd1765223ec81816ac56b8f202ac
Sep 10 20:59:46.513659 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.513639    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-947bp\""
Sep 10 20:59:46.520337 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.520318    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:46.795037 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.795007    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r86bc"]
Sep 10 20:59:46.797870 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:46.797841    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b49b59b_81ba_4c02_851c_3cfa4db12c79.slice/crio-4cb13531af2d3a3c950dbf2dae9a9af0e6c253e8d43de7a354b047f449d9aee8 WatchSource:0}: Error finding container 4cb13531af2d3a3c950dbf2dae9a9af0e6c253e8d43de7a354b047f449d9aee8: Status 404 returned error can't find the container with id 4cb13531af2d3a3c950dbf2dae9a9af0e6c253e8d43de7a354b047f449d9aee8
Sep 10 20:59:46.883901 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.883863    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r86bc" event={"ID":"0b49b59b-81ba-4c02-851c-3cfa4db12c79","Type":"ContainerStarted","Data":"651f9149c8a686005c4c46e785babe3e712285c192b9817b1e0e6dfc4f2e06f8"}
Sep 10 20:59:46.884342 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.883924    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r86bc" event={"ID":"0b49b59b-81ba-4c02-851c-3cfa4db12c79","Type":"ContainerStarted","Data":"4cb13531af2d3a3c950dbf2dae9a9af0e6c253e8d43de7a354b047f449d9aee8"}
Sep 10 20:59:46.884342 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.883961    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 20:59:46.885555 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.885495    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r6rnh" event={"ID":"f171cba7-5c57-4da6-a497-750d801b1ce2","Type":"ContainerStarted","Data":"7a81d07079bf736d6322ccb675601ca0a8290525dd2e3cf54e6ecb0a61fa1329"}
Sep 10 20:59:46.887858 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.887834    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lkmvb" event={"ID":"c934801f-9482-4a87-b9d7-a153a5be0e5e","Type":"ContainerStarted","Data":"42ed7ca7d8223d75fa4950aea20ed997aaaefd1765223ec81816ac56b8f202ac"}
Sep 10 20:59:46.889828 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.889798    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6chtx" event={"ID":"5d19bacb-b122-4c14-8f38-99989bc66f71","Type":"ContainerStarted","Data":"4369727f4da7c7cfc5e79cd7472b4aed8d68bbf445d0070d4401df3ad2b85953"}
Sep 10 20:59:46.940011 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:46.939841    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r86bc" podStartSLOduration=66.939823547 podStartE2EDuration="1m6.939823547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:46.939219877 +0000 UTC m=+66.958625216" watchObservedRunningTime="2026-09-10 20:59:46.939823547 +0000 UTC m=+66.959228866"
Sep 10 20:59:47.035771 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.035733    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:47.035929 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.035802    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:47.038834 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.038779    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"image-registry-69f66476c-vtvls\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") " pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:47.041276 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.041229    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0fdee397-ee70-4c4f-87e3-95289083c592-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-ghgxz\" (UID: \"0fdee397-ee70-4c4f-87e3-95289083c592\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:47.277063 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.276902    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t568q\""
Sep 10 20:59:47.281825 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.281797    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"
Sep 10 20:59:47.304805 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.304596    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bbgkn\""
Sep 10 20:59:47.312143 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.311915    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:47.749872 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.749827    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69f66476c-vtvls"]
Sep 10 20:59:47.756122 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.756025    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz"]
Sep 10 20:59:47.895769 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:47.895610    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lkmvb" event={"ID":"c934801f-9482-4a87-b9d7-a153a5be0e5e","Type":"ContainerStarted","Data":"69480aed142ebe32a063b6c629d38ca024cf9b94037eb65dcf165b6acf14772a"}
Sep 10 20:59:48.144147 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:48.144107    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc909cd_c0dc_4b77_b0fd_9bcf2473f59b.slice/crio-6535caa41b574304a219f172716c528d2e678c9ed2bb638cca9f93524ee61c25 WatchSource:0}: Error finding container 6535caa41b574304a219f172716c528d2e678c9ed2bb638cca9f93524ee61c25: Status 404 returned error can't find the container with id 6535caa41b574304a219f172716c528d2e678c9ed2bb638cca9f93524ee61c25
Sep 10 20:59:48.145054 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:48.145013    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fdee397_ee70_4c4f_87e3_95289083c592.slice/crio-bc4ade2ab6ed9266f47fc02cc5ce8ca7415a0a2b75ced3d7f09f26b6d6a98d8e WatchSource:0}: Error finding container bc4ade2ab6ed9266f47fc02cc5ce8ca7415a0a2b75ced3d7f09f26b6d6a98d8e: Status 404 returned error can't find the container with id bc4ade2ab6ed9266f47fc02cc5ce8ca7415a0a2b75ced3d7f09f26b6d6a98d8e
Sep 10 20:59:48.902567 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.902529    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f66476c-vtvls" event={"ID":"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b","Type":"ContainerStarted","Data":"6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4"}
Sep 10 20:59:48.903059 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.902590    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f66476c-vtvls" event={"ID":"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b","Type":"ContainerStarted","Data":"6535caa41b574304a219f172716c528d2e678c9ed2bb638cca9f93524ee61c25"}
Sep 10 20:59:48.903059 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.902740    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 20:59:48.904448 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.904397    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz" event={"ID":"0fdee397-ee70-4c4f-87e3-95289083c592","Type":"ContainerStarted","Data":"bc4ade2ab6ed9266f47fc02cc5ce8ca7415a0a2b75ced3d7f09f26b6d6a98d8e"}
Sep 10 20:59:48.907153 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.906857    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lkmvb" event={"ID":"c934801f-9482-4a87-b9d7-a153a5be0e5e","Type":"ContainerStarted","Data":"1a1c0d7ab46021b03deb128bf17d3c4a2736c97877031065331edb7e3056df44"}
Sep 10 20:59:48.909371 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.909347    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r6rnh" event={"ID":"f171cba7-5c57-4da6-a497-750d801b1ce2","Type":"ContainerStarted","Data":"537996558d2b4b7e111ccf9cf21a453c69ea04af9713c63baa2be2d5bedf044e"}
Sep 10 20:59:48.909506 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.909406    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r6rnh" event={"ID":"f171cba7-5c57-4da6-a497-750d801b1ce2","Type":"ContainerStarted","Data":"dc98a9f24ba23db706e339416e1001289d9cef97c11f22b8054b314d41cdb6eb"}
Sep 10 20:59:48.909506 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.909457    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:48.952317 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.952265    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69f66476c-vtvls" podStartSLOduration=33.952244698 podStartE2EDuration="33.952244698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:48.951582328 +0000 UTC m=+68.970987647" watchObservedRunningTime="2026-09-10 20:59:48.952244698 +0000 UTC m=+68.971650016"
Sep 10 20:59:48.985821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:48.985778    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lkmvb" podStartSLOduration=68.01337316 podStartE2EDuration="1m8.985761505s" totalImagesPullingTime="972.388345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:48.985398164 +0000 UTC m=+69.004803484" watchObservedRunningTime="2026-09-10 20:59:48.985761505 +0000 UTC m=+69.005166856"
Sep 10 20:59:49.016675 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:49.016581    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r6rnh" podStartSLOduration=34.235085394 podStartE2EDuration="36.016565065s" totalImagesPullingTime="1.781479671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:49.016319414 +0000 UTC m=+69.035724733" watchObservedRunningTime="2026-09-10 20:59:49.016565065 +0000 UTC m=+69.035970385"
Sep 10 20:59:51.922061 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:51.922018    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz" event={"ID":"0fdee397-ee70-4c4f-87e3-95289083c592","Type":"ContainerStarted","Data":"f566435fdb35532d458d9a94786fe5f91e2f71468eb11006c3a5d3e88944943a"}
Sep 10 20:59:51.923645 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:51.923621    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6chtx" event={"ID":"5d19bacb-b122-4c14-8f38-99989bc66f71","Type":"ContainerStarted","Data":"0d346cd710abeab4872d1b96760b39d023340253ad0fff34e861ee07e8b26027"}
Sep 10 20:59:51.951974 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:51.951932    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-ghgxz" podStartSLOduration=34.002712781 podStartE2EDuration="36.951920499s" totalImagesPullingTime="2.949207718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:51.950729401 +0000 UTC m=+71.970134720" watchObservedRunningTime="2026-09-10 20:59:51.951920499 +0000 UTC m=+71.971325816"
Sep 10 20:59:51.977287 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:51.977242    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6chtx" podStartSLOduration=67.373704166 podStartE2EDuration="1m11.97722444s" totalImagesPullingTime="4.603520274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:58:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:51.97627072 +0000 UTC m=+71.995676038" watchObservedRunningTime="2026-09-10 20:59:51.97722444 +0000 UTC m=+71.996629759"
Sep 10 20:59:52.625390 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.625360    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-jh6c8"]
Sep 10 20:59:52.657721 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.657666    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69f66476c-vtvls"]
Sep 10 20:59:52.657882 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.657811    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-jh6c8"
Sep 10 20:59:52.666797 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.666769    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-jh6c8"]
Sep 10 20:59:52.667776 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.667755    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-5ns84\""
Sep 10 20:59:52.668489 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.668460    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 20:59:52.668746 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.668588    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 20:59:52.687416 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.687394    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"]
Sep 10 20:59:52.711850 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.711829    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"
Sep 10 20:59:52.715108 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.715089    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-22sj2\""
Sep 10 20:59:52.715417 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.715401    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 10 20:59:52.742730 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.742706    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58c866cf74-f6sw9"]
Sep 10 20:59:52.763829 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.763805    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"]
Sep 10 20:59:52.763943 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.763893    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.775717 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.775696    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c866cf74-f6sw9"]
Sep 10 20:59:52.785002 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.784979    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d069a24-c52a-4e12-8bcd-edc94386a876-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-zr49p\" (UID: \"6d069a24-c52a-4e12-8bcd-edc94386a876\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"
Sep 10 20:59:52.785099 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.785017    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b4n9\" (UniqueName: \"kubernetes.io/projected/a44413b6-44a3-4b54-ad71-14348181926e-kube-api-access-2b4n9\") pod \"downloads-76f99584f4-jh6c8\" (UID: \"a44413b6-44a3-4b54-ad71-14348181926e\") " pod="openshift-console/downloads-76f99584f4-jh6c8"
Sep 10 20:59:52.886394 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886316    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tx77\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-kube-api-access-4tx77\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.886394 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73c3a136-cde8-4373-a1ef-01124e302c3b-installation-pull-secrets\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.886394 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886382    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d069a24-c52a-4e12-8bcd-edc94386a876-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-zr49p\" (UID: \"6d069a24-c52a-4e12-8bcd-edc94386a876\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"
Sep 10 20:59:52.886594 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886503    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73c3a136-cde8-4373-a1ef-01124e302c3b-image-registry-private-configuration\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.886594 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886532    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-bound-sa-token\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.886594 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886560    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73c3a136-cde8-4373-a1ef-01124e302c3b-trusted-ca\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.886594 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886578    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73c3a136-cde8-4373-a1ef-01124e302c3b-ca-trust-extracted\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.886757 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886621    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b4n9\" (UniqueName: \"kubernetes.io/projected/a44413b6-44a3-4b54-ad71-14348181926e-kube-api-access-2b4n9\") pod \"downloads-76f99584f4-jh6c8\" (UID: \"a44413b6-44a3-4b54-ad71-14348181926e\") " pod="openshift-console/downloads-76f99584f4-jh6c8"
Sep 10 20:59:52.886757 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886695    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73c3a136-cde8-4373-a1ef-01124e302c3b-registry-certificates\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.886757 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.886739    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-registry-tls\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.888787 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.888769    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/6d069a24-c52a-4e12-8bcd-edc94386a876-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-zr49p\" (UID: \"6d069a24-c52a-4e12-8bcd-edc94386a876\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"
Sep 10 20:59:52.963102 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.963070    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b4n9\" (UniqueName: \"kubernetes.io/projected/a44413b6-44a3-4b54-ad71-14348181926e-kube-api-access-2b4n9\") pod \"downloads-76f99584f4-jh6c8\" (UID: \"a44413b6-44a3-4b54-ad71-14348181926e\") " pod="openshift-console/downloads-76f99584f4-jh6c8"
Sep 10 20:59:52.967678 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.967652    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-jh6c8"
Sep 10 20:59:52.987490 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987463    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tx77\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-kube-api-access-4tx77\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.987604 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987499    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73c3a136-cde8-4373-a1ef-01124e302c3b-installation-pull-secrets\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.987604 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987523    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73c3a136-cde8-4373-a1ef-01124e302c3b-image-registry-private-configuration\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.987604 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987540    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-bound-sa-token\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.987604 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987593    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73c3a136-cde8-4373-a1ef-01124e302c3b-trusted-ca\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.987821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987621    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73c3a136-cde8-4373-a1ef-01124e302c3b-ca-trust-extracted\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.987821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987694    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73c3a136-cde8-4373-a1ef-01124e302c3b-registry-certificates\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.987821 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.987738    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-registry-tls\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.988264 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.988229    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73c3a136-cde8-4373-a1ef-01124e302c3b-ca-trust-extracted\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.988658 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.988629    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73c3a136-cde8-4373-a1ef-01124e302c3b-registry-certificates\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.988910 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.988891    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73c3a136-cde8-4373-a1ef-01124e302c3b-trusted-ca\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.990261 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.990239    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-registry-tls\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.990510 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.990490    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73c3a136-cde8-4373-a1ef-01124e302c3b-installation-pull-secrets\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:52.990625 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:52.990602    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73c3a136-cde8-4373-a1ef-01124e302c3b-image-registry-private-configuration\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:53.020046 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.020012    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"
Sep 10 20:59:53.054863 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.054840    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-bound-sa-token\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:53.059366 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.059340    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tx77\" (UniqueName: \"kubernetes.io/projected/73c3a136-cde8-4373-a1ef-01124e302c3b-kube-api-access-4tx77\") pod \"image-registry-58c866cf74-f6sw9\" (UID: \"73c3a136-cde8-4373-a1ef-01124e302c3b\") " pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:53.072478 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.072454    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:53.189136 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.189094    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:53.191329 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.191308    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04bdb162-6a86-43fc-a5b7-f3de85e4b1e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cfvm7\" (UID: \"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5\") " pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:53.284290 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:53.284261    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44413b6_44a3_4b54_ad71_14348181926e.slice/crio-5a743f29fa256a5ce6a6704fa19017604a4021bdc42dd85317128473101068bd WatchSource:0}: Error finding container 5a743f29fa256a5ce6a6704fa19017604a4021bdc42dd85317128473101068bd: Status 404 returned error can't find the container with id 5a743f29fa256a5ce6a6704fa19017604a4021bdc42dd85317128473101068bd
Sep 10 20:59:53.286419 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.286392    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-jh6c8"]
Sep 10 20:59:53.405292 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.405205    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"]
Sep 10 20:59:53.412240 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.412217    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7vxwx\""
Sep 10 20:59:53.420144 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.420120    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cfvm7"
Sep 10 20:59:53.426525 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:53.426499    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d069a24_c52a_4e12_8bcd_edc94386a876.slice/crio-f62083f8e5b124e6b720bd7c9f660752ae460fa2fc71630a6be2f6076aaf2a9f WatchSource:0}: Error finding container f62083f8e5b124e6b720bd7c9f660752ae460fa2fc71630a6be2f6076aaf2a9f: Status 404 returned error can't find the container with id f62083f8e5b124e6b720bd7c9f660752ae460fa2fc71630a6be2f6076aaf2a9f
Sep 10 20:59:53.549912 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.549878    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c866cf74-f6sw9"]
Sep 10 20:59:53.553032 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:53.553008    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c3a136_cde8_4373_a1ef_01124e302c3b.slice/crio-3aa345ed9e0f3794e0dfc0f99cb7660b73ff6b3e5671a7850b265f0a651f46ef WatchSource:0}: Error finding container 3aa345ed9e0f3794e0dfc0f99cb7660b73ff6b3e5671a7850b265f0a651f46ef: Status 404 returned error can't find the container with id 3aa345ed9e0f3794e0dfc0f99cb7660b73ff6b3e5671a7850b265f0a651f46ef
Sep 10 20:59:53.745789 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.745760    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cfvm7"]
Sep 10 20:59:53.749028 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:53.749001    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04bdb162_6a86_43fc_a5b7_f3de85e4b1e5.slice/crio-cfd8d722d2c0df6c844384a428b5de27a652b5a162e5d2b24ff509c4afde7cf4 WatchSource:0}: Error finding container cfd8d722d2c0df6c844384a428b5de27a652b5a162e5d2b24ff509c4afde7cf4: Status 404 returned error can't find the container with id cfd8d722d2c0df6c844384a428b5de27a652b5a162e5d2b24ff509c4afde7cf4
Sep 10 20:59:53.934721 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.934615    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c866cf74-f6sw9" event={"ID":"73c3a136-cde8-4373-a1ef-01124e302c3b","Type":"ContainerStarted","Data":"dbf5213ef9f61270507ded222ab69855fb5d450e7a9144269cfd24dc98751eb3"}
Sep 10 20:59:53.934721 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.934701    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c866cf74-f6sw9" event={"ID":"73c3a136-cde8-4373-a1ef-01124e302c3b","Type":"ContainerStarted","Data":"3aa345ed9e0f3794e0dfc0f99cb7660b73ff6b3e5671a7850b265f0a651f46ef"}
Sep 10 20:59:53.934925 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.934728    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 20:59:53.936279 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.936234    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p" event={"ID":"6d069a24-c52a-4e12-8bcd-edc94386a876","Type":"ContainerStarted","Data":"f62083f8e5b124e6b720bd7c9f660752ae460fa2fc71630a6be2f6076aaf2a9f"}
Sep 10 20:59:53.937782 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.937747    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-jh6c8" event={"ID":"a44413b6-44a3-4b54-ad71-14348181926e","Type":"ContainerStarted","Data":"5a743f29fa256a5ce6a6704fa19017604a4021bdc42dd85317128473101068bd"}
Sep 10 20:59:53.939487 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.939463    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfvm7" event={"ID":"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5","Type":"ContainerStarted","Data":"122a02cede7d6e65ce12c679515f8b382bb306eecf287bd984a8696425cb880a"}
Sep 10 20:59:53.939586 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.939515    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfvm7" event={"ID":"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5","Type":"ContainerStarted","Data":"cfd8d722d2c0df6c844384a428b5de27a652b5a162e5d2b24ff509c4afde7cf4"}
Sep 10 20:59:53.988826 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:53.988773    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58c866cf74-f6sw9" podStartSLOduration=1.988753365 podStartE2EDuration="1.988753365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:53.988239231 +0000 UTC m=+74.007644549" watchObservedRunningTime="2026-09-10 20:59:53.988753365 +0000 UTC m=+74.008158718"
Sep 10 20:59:54.945993 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:54.945951    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p" event={"ID":"6d069a24-c52a-4e12-8bcd-edc94386a876","Type":"ContainerStarted","Data":"b9c10e38937234a88e3a517acb2e5e407ed2c482630fd970a7bfb869c621e593"}
Sep 10 20:59:54.946201 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:54.946150    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"
Sep 10 20:59:54.948606 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:54.948557    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfvm7" event={"ID":"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5","Type":"ContainerStarted","Data":"8e4cf4c736f76c73b40b19ce05e92ef55af6b43ed270bde769857c39d6ad737b"}
Sep 10 20:59:54.951883 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:54.951861    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p"
Sep 10 20:59:54.971934 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:54.971887    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-zr49p" podStartSLOduration=1.618205009 podStartE2EDuration="2.971873504s" totalImagesPullingTime="1.353668495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:54.970865475 +0000 UTC m=+74.990270793" watchObservedRunningTime="2026-09-10 20:59:54.971873504 +0000 UTC m=+74.991278822"
Sep 10 20:59:55.904831 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:55.904714    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"]
Sep 10 20:59:55.907057 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:55.907031    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:55.911142 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:55.911115    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 10 20:59:55.911274 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:55.911247    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xdkbg\""
Sep 10 20:59:55.912294 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:55.912274    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 20:59:55.915430 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:55.915291    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 10 20:59:55.929637 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:55.929571    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"]
Sep 10 20:59:56.011727 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.011666    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vcd\" (UniqueName: \"kubernetes.io/projected/3648aac7-e065-4b16-b52b-66fe5641da9a-kube-api-access-z2vcd\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.011889 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.011798    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3648aac7-e065-4b16-b52b-66fe5641da9a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.011889 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.011877    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.012005 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.011950    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.113698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.112969    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3648aac7-e065-4b16-b52b-66fe5641da9a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.113698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.113060    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.113698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.113137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.113698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.113171    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vcd\" (UniqueName: \"kubernetes.io/projected/3648aac7-e065-4b16-b52b-66fe5641da9a-kube-api-access-z2vcd\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.113698 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.113701    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3648aac7-e065-4b16-b52b-66fe5641da9a-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.114076 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:56.113803    2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 10 20:59:56.114076 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:56.113865    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls podName:3648aac7-e065-4b16-b52b-66fe5641da9a nodeName:}" failed. No retries permitted until 2026-09-10 20:59:56.613845536 +0000 UTC m=+76.633250838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-l2qn9" (UID: "3648aac7-e065-4b16-b52b-66fe5641da9a") : secret "prometheus-operator-tls" not found
Sep 10 20:59:56.116491 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.116405    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.126076 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.126030    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vcd\" (UniqueName: \"kubernetes.io/projected/3648aac7-e065-4b16-b52b-66fe5641da9a-kube-api-access-z2vcd\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.617608 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.617575    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:56.617772 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:56.617744    2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 10 20:59:56.617817 ip-10-0-140-211 kubenswrapper[2557]: E0910 20:59:56.617804    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls podName:3648aac7-e065-4b16-b52b-66fe5641da9a nodeName:}" failed. No retries permitted until 2026-09-10 20:59:57.617790154 +0000 UTC m=+77.637195454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls") pod "prometheus-operator-68bcd7f987-l2qn9" (UID: "3648aac7-e065-4b16-b52b-66fe5641da9a") : secret "prometheus-operator-tls" not found
Sep 10 20:59:56.958206 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.958127    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cfvm7" event={"ID":"04bdb162-6a86-43fc-a5b7-f3de85e4b1e5","Type":"ContainerStarted","Data":"89e3c7ebf43658b6b9d0b974dd81890171f957d914534ad2ceaaecdcad737032"}
Sep 10 20:59:56.993464 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:56.993404    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cfvm7" podStartSLOduration=33.469661769 podStartE2EDuration="35.993388176s" totalImagesPullingTime="2.523726407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 20:59:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 20:59:56.991535706 +0000 UTC m=+77.010941023" watchObservedRunningTime="2026-09-10 20:59:56.993388176 +0000 UTC m=+77.012793494"
Sep 10 20:59:57.624778 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:57.624737    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:57.627384 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:57.627357    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3648aac7-e065-4b16-b52b-66fe5641da9a-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-l2qn9\" (UID: \"3648aac7-e065-4b16-b52b-66fe5641da9a\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:57.718341 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:57.718304    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"
Sep 10 20:59:58.001940 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:58.001909    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9"]
Sep 10 20:59:58.005243 ip-10-0-140-211 kubenswrapper[2557]: W0910 20:59:58.005211    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3648aac7_e065_4b16_b52b_66fe5641da9a.slice/crio-8ed6c14e2c1b3cbb910a662548bce045222b8dd25c470ce79135a2b3b1578888 WatchSource:0}: Error finding container 8ed6c14e2c1b3cbb910a662548bce045222b8dd25c470ce79135a2b3b1578888: Status 404 returned error can't find the container with id 8ed6c14e2c1b3cbb910a662548bce045222b8dd25c470ce79135a2b3b1578888
Sep 10 20:59:58.915741 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:58.915705    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r6rnh"
Sep 10 20:59:58.968202 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:58.968165    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9" event={"ID":"3648aac7-e065-4b16-b52b-66fe5641da9a","Type":"ContainerStarted","Data":"8ed6c14e2c1b3cbb910a662548bce045222b8dd25c470ce79135a2b3b1578888"}
Sep 10 20:59:59.974241 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:59.974168    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9" event={"ID":"3648aac7-e065-4b16-b52b-66fe5641da9a","Type":"ContainerStarted","Data":"06ea8f71bca61ddc8354851df3848254c6941611954f637de2553d80f0963235"}
Sep 10 20:59:59.974241 ip-10-0-140-211 kubenswrapper[2557]: I0910 20:59:59.974227    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9" event={"ID":"3648aac7-e065-4b16-b52b-66fe5641da9a","Type":"ContainerStarted","Data":"ecc835ce75edc8ccf65264f4783f1bff6e378515213ae1fc5f0f3486667bdc20"}
Sep 10 21:00:00.015273 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:00.015207    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-l2qn9" podStartSLOduration=3.303891202 podStartE2EDuration="5.015187316s" totalImagesPullingTime="1.711296114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:00.014451387 +0000 UTC m=+80.033856738" watchObservedRunningTime="2026-09-10 21:00:00.015187316 +0000 UTC m=+80.034592642"
Sep 10 21:00:02.506441 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.506264    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"]
Sep 10 21:00:02.511105 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.511028    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.514694 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:00:02.514636    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"openshift-state-metrics-67dcf8c979-45spn\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.514842 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:00:02.514807    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"openshift-state-metrics-dockercfg-g568d\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-g568d\"" type="*v1.Secret"
Sep 10 21:00:02.518325 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:00:02.516642    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"openshift-state-metrics-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" type="*v1.Secret"
Sep 10 21:00:02.518325 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:00:02.516727    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"openshift-state-metrics-tls\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" type="*v1.Secret"
Sep 10 21:00:02.518325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.516915    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"]
Sep 10 21:00:02.522254 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.522196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.525515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.525492    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"]
Sep 10 21:00:02.526265 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.526242    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hntj4\""
Sep 10 21:00:02.531289 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.531260    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 21:00:02.531482 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.531462    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 21:00:02.531773 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.531756    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 21:00:02.539559 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.539538    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"]
Sep 10 21:00:02.541555 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.541532    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5c4zz"]
Sep 10 21:00:02.545925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.545126    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.555505 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.555484    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 21:00:02.555721 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.555699    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 21:00:02.555812 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.555779    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m26fb\""
Sep 10 21:00:02.556013 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.555996    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 21:00:02.670677 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670638    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f4c0123-ee19-4773-89af-479ff47e1562-metrics-client-ca\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.670871 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670710    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.670871 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670742    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62ba902d-4465-48d9-b9c8-d907423c9ee8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.670871 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670786    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.670871 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670813    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-tls\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.671097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670867    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgj6m\" (UniqueName: \"kubernetes.io/projected/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-api-access-jgj6m\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.671097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670951    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-root\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.671097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.670999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.671097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671034    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.671097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671073    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671104    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-wtmp\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671138    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7n54\" (UniqueName: \"kubernetes.io/projected/3f4c0123-ee19-4773-89af-479ff47e1562-kube-api-access-f7n54\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671190    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ba902d-4465-48d9-b9c8-d907423c9ee8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671229    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-sys\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671288    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671313    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-textfile\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.671357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671340    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-accelerators-collector-config\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.671734 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.671365    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwqpp\" (UniqueName: \"kubernetes.io/projected/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-kube-api-access-wwqpp\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.772335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772240    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.772335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772290    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-tls\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.772335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgj6m\" (UniqueName: \"kubernetes.io/projected/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-api-access-jgj6m\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.772565 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772346    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-root\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.772565 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772410    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-root\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.772565 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772526    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.772699 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772564    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.772699 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772602    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.772699 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772634    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-wtmp\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.772699 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772666    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7n54\" (UniqueName: \"kubernetes.io/projected/3f4c0123-ee19-4773-89af-479ff47e1562-kube-api-access-f7n54\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.772848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772712    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ba902d-4465-48d9-b9c8-d907423c9ee8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.772848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772743    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.772848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772764    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-sys\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.772848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772809    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.772848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772831    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-textfile\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.773094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772854    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-accelerators-collector-config\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.773094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772879    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwqpp\" (UniqueName: \"kubernetes.io/projected/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-kube-api-access-wwqpp\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.773094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772912    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f4c0123-ee19-4773-89af-479ff47e1562-metrics-client-ca\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.773094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772936    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.773094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.772962    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62ba902d-4465-48d9-b9c8-d907423c9ee8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.773311 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.773206    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-wtmp\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.773366 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.773344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62ba902d-4465-48d9-b9c8-d907423c9ee8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.773410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.773381    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.774215 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.773509    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-textfile\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.774215 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.773575    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f4c0123-ee19-4773-89af-479ff47e1562-sys\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.774215 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.773985    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-accelerators-collector-config\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.774215 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.773993    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f4c0123-ee19-4773-89af-479ff47e1562-metrics-client-ca\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.774215 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.774154    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62ba902d-4465-48d9-b9c8-d907423c9ee8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.774528 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.774287    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.775270 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.775250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-tls\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.776172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.776148    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.776262 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.776183    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.776665 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.776546    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f4c0123-ee19-4773-89af-479ff47e1562-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.787592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.787568    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7n54\" (UniqueName: \"kubernetes.io/projected/3f4c0123-ee19-4773-89af-479ff47e1562-kube-api-access-f7n54\") pod \"node-exporter-5c4zz\" (UID: \"3f4c0123-ee19-4773-89af-479ff47e1562\") " pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.788315 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.788285    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgj6m\" (UniqueName: \"kubernetes.io/projected/62ba902d-4465-48d9-b9c8-d907423c9ee8-kube-api-access-jgj6m\") pod \"kube-state-metrics-59fbdd77d-rhkk4\" (UID: \"62ba902d-4465-48d9-b9c8-d907423c9ee8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.788826 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.788803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwqpp\" (UniqueName: \"kubernetes.io/projected/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-kube-api-access-wwqpp\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:02.842080 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.842054    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"
Sep 10 21:00:02.863581 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.863558    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5c4zz"
Sep 10 21:00:02.882584 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:00:02.882548    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f4c0123_ee19_4773_89af_479ff47e1562.slice/crio-5e63af3ac3c107f0722259a308df4b5fc183c8b77ee6d1e1a8b61356382e14cc WatchSource:0}: Error finding container 5e63af3ac3c107f0722259a308df4b5fc183c8b77ee6d1e1a8b61356382e14cc: Status 404 returned error can't find the container with id 5e63af3ac3c107f0722259a308df4b5fc183c8b77ee6d1e1a8b61356382e14cc
Sep 10 21:00:02.989540 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:02.989507    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5c4zz" event={"ID":"3f4c0123-ee19-4773-89af-479ff47e1562","Type":"ContainerStarted","Data":"5e63af3ac3c107f0722259a308df4b5fc183c8b77ee6d1e1a8b61356382e14cc"}
Sep 10 21:00:03.163102 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:03.163023    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4"]
Sep 10 21:00:03.547276 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:03.547238    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-g568d\""
Sep 10 21:00:03.575501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:03.575467    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 10 21:00:03.586166 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:03.586138    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:03.641725 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:03.641678    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 10 21:00:03.646545 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:03.646518    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-45spn\" (UID: \"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:03.727826 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:03.727702    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"
Sep 10 21:00:07.719238 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.719196    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78f977ff8f-krggd"]
Sep 10 21:00:07.723024 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.722999    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.732328 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.732306    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 10 21:00:07.733048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.733016    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 10 21:00:07.733438 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.733417    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 10 21:00:07.733550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.733422    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 10 21:00:07.733796 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.733769    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5z2wk\""
Sep 10 21:00:07.733893 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.733770    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 10 21:00:07.737300 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.737280    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 10 21:00:07.758404 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.758376    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78f977ff8f-krggd"]
Sep 10 21:00:07.818755 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.818720    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-secret-telemeter-client\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.818929 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.818785    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shmdl\" (UniqueName: \"kubernetes.io/projected/d65230cd-1a5d-44e2-8d16-216d529dc190-kube-api-access-shmdl\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.818929 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.818873    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-metrics-client-ca\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.819048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.818928    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-telemeter-client-tls\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.819048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.818957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.819048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.818988    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-serving-certs-ca-bundle\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.819048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.819031    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.819200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.819053    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-federate-client-tls\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920015 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.919977    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-telemeter-client-tls\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920021    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920051    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-serving-certs-ca-bundle\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920089    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920116    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-federate-client-tls\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920146    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-secret-telemeter-client\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920194    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shmdl\" (UniqueName: \"kubernetes.io/projected/d65230cd-1a5d-44e2-8d16-216d529dc190-kube-api-access-shmdl\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.920494 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920230    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-metrics-client-ca\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.921043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.920990    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-metrics-client-ca\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.921187 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.921128    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.921754 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.921708    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d65230cd-1a5d-44e2-8d16-216d529dc190-serving-certs-ca-bundle\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.923110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.923085    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.923220 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.923175    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-federate-client-tls\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.923306 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.923243    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-secret-telemeter-client\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.923306 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.923256    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d65230cd-1a5d-44e2-8d16-216d529dc190-telemeter-client-tls\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:07.934543 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:07.934516    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shmdl\" (UniqueName: \"kubernetes.io/projected/d65230cd-1a5d-44e2-8d16-216d529dc190-kube-api-access-shmdl\") pod \"telemeter-client-78f977ff8f-krggd\" (UID: \"d65230cd-1a5d-44e2-8d16-216d529dc190\") " pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:08.034827 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:08.034742    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd"
Sep 10 21:00:10.235902 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:00:10.235869    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ba902d_4465_48d9_b9c8_d907423c9ee8.slice/crio-3cbd5845edfc187cbd5d9f33a1a5ff0a40617ad574eb92001e9de3cb53f8fd02 WatchSource:0}: Error finding container 3cbd5845edfc187cbd5d9f33a1a5ff0a40617ad574eb92001e9de3cb53f8fd02: Status 404 returned error can't find the container with id 3cbd5845edfc187cbd5d9f33a1a5ff0a40617ad574eb92001e9de3cb53f8fd02
Sep 10 21:00:10.529714 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:10.529654    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn"]
Sep 10 21:00:10.531360 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:00:10.531328    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode998b1ee_76b5_4fe5_a2ea_d14fd0c43b74.slice/crio-b4157c49608f74e6f51dd0404a245afd256c6239463794558cdceb5248db8799 WatchSource:0}: Error finding container b4157c49608f74e6f51dd0404a245afd256c6239463794558cdceb5248db8799: Status 404 returned error can't find the container with id b4157c49608f74e6f51dd0404a245afd256c6239463794558cdceb5248db8799
Sep 10 21:00:10.543851 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:10.543810    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78f977ff8f-krggd"]
Sep 10 21:00:10.548875 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:00:10.548837    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65230cd_1a5d_44e2_8d16_216d529dc190.slice/crio-f6946c881e7ce6f59ee8ae093a2ccba53f334accfee89dcd863f327188f42234 WatchSource:0}: Error finding container f6946c881e7ce6f59ee8ae093a2ccba53f334accfee89dcd863f327188f42234: Status 404 returned error can't find the container with id f6946c881e7ce6f59ee8ae093a2ccba53f334accfee89dcd863f327188f42234
Sep 10 21:00:11.024323 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.024276    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd" event={"ID":"d65230cd-1a5d-44e2-8d16-216d529dc190","Type":"ContainerStarted","Data":"f6946c881e7ce6f59ee8ae093a2ccba53f334accfee89dcd863f327188f42234"}
Sep 10 21:00:11.026446 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.026398    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-jh6c8" event={"ID":"a44413b6-44a3-4b54-ad71-14348181926e","Type":"ContainerStarted","Data":"e8ccb385ff6bd17c82d79eca27b46a693b15d1e35da6204c0576947786b0b88f"}
Sep 10 21:00:11.026813 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.026740    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-jh6c8"
Sep 10 21:00:11.028805 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.028780    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn" event={"ID":"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74","Type":"ContainerStarted","Data":"8ea988f88a0dacc249f76db7547893cba8a33ff8735c6a3dda9d65f95341c44b"}
Sep 10 21:00:11.028899 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.028828    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn" event={"ID":"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74","Type":"ContainerStarted","Data":"820fb3996b6fe31c5a903890abdf25768468129d78fc8e81c9d5ee99b6c010ab"}
Sep 10 21:00:11.028899 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.028857    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn" event={"ID":"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74","Type":"ContainerStarted","Data":"b4157c49608f74e6f51dd0404a245afd256c6239463794558cdceb5248db8799"}
Sep 10 21:00:11.030842 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.030811    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5c4zz" event={"ID":"3f4c0123-ee19-4773-89af-479ff47e1562","Type":"ContainerStarted","Data":"444bf25ba9e0f32f051f7c2d58266217d20a209f1a7b79ca3431d959a722c1fe"}
Sep 10 21:00:11.032396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.032332    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4" event={"ID":"62ba902d-4465-48d9-b9c8-d907423c9ee8","Type":"ContainerStarted","Data":"3cbd5845edfc187cbd5d9f33a1a5ff0a40617ad574eb92001e9de3cb53f8fd02"}
Sep 10 21:00:11.043103 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.042955    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-jh6c8"
Sep 10 21:00:11.053741 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:11.053601    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-jh6c8" podStartSLOduration=2.002007859 podStartE2EDuration="19.053582469s" totalImagesPullingTime="17.05157461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 20:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:11.051433121 +0000 UTC m=+91.070838440" watchObservedRunningTime="2026-09-10 21:00:11.053582469 +0000 UTC m=+91.072987788"
Sep 10 21:00:12.044711 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:12.044035    2557 generic.go:356] "Generic (PLEG): container finished" podID="3f4c0123-ee19-4773-89af-479ff47e1562" containerID="444bf25ba9e0f32f051f7c2d58266217d20a209f1a7b79ca3431d959a722c1fe" exitCode=0
Sep 10 21:00:12.044711 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:12.044147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5c4zz" event={"ID":"3f4c0123-ee19-4773-89af-479ff47e1562","Type":"ContainerDied","Data":"444bf25ba9e0f32f051f7c2d58266217d20a209f1a7b79ca3431d959a722c1fe"}
Sep 10 21:00:12.664910 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:12.664874    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 21:00:14.059885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:14.059619    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5c4zz" event={"ID":"3f4c0123-ee19-4773-89af-479ff47e1562","Type":"ContainerStarted","Data":"d01bd7ad3ade1e12b0ece650ceda616407a10037eb682bd9949338ebbbdfb22a"}
Sep 10 21:00:14.061647 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:14.061337    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4" event={"ID":"62ba902d-4465-48d9-b9c8-d907423c9ee8","Type":"ContainerStarted","Data":"449770c58f2cdf04105f670c81c99d76320338a6b81f993a7b627616e91dab55"}
Sep 10 21:00:14.064636 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:14.063051    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd" event={"ID":"d65230cd-1a5d-44e2-8d16-216d529dc190","Type":"ContainerStarted","Data":"42e9c7abac58fab49d361bd85629bead39028ea000f54f9b2cd68f29681c53c2"}
Sep 10 21:00:14.065360 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:14.065339    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn" event={"ID":"e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74","Type":"ContainerStarted","Data":"7a075996c129f44c846d8d229112a3136c9acd558bb4e4daa81ece9d113cf495"}
Sep 10 21:00:14.108860 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:14.108796    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-45spn" podStartSLOduration=9.208954439 podStartE2EDuration="12.108773725s" totalImagesPullingTime="2.899819286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:14.106297176 +0000 UTC m=+94.125702497" watchObservedRunningTime="2026-09-10 21:00:14.108773725 +0000 UTC m=+94.128179064"
Sep 10 21:00:14.954217 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:14.954183    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58c866cf74-f6sw9"
Sep 10 21:00:15.075146 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:15.075083    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5c4zz" event={"ID":"3f4c0123-ee19-4773-89af-479ff47e1562","Type":"ContainerStarted","Data":"ad020de84c6ccfcb08a250a8ecf44033719f2756b604dcb3c7d91d794dcc7fdc"}
Sep 10 21:00:15.077747 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:15.077718    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4" event={"ID":"62ba902d-4465-48d9-b9c8-d907423c9ee8","Type":"ContainerStarted","Data":"a8ebb43e9402cc148544859b69c53202e925f075649bdd2d00a00c29a676a976"}
Sep 10 21:00:15.077881 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:15.077778    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4" event={"ID":"62ba902d-4465-48d9-b9c8-d907423c9ee8","Type":"ContainerStarted","Data":"5816604d85aa5a8a3d42dcbd2196cf1dc0650ad313f7de07622848d36da1de69"}
Sep 10 21:00:15.137778 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:15.137706    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5c4zz" podStartSLOduration=5.03526124 podStartE2EDuration="13.137672505s" totalImagesPullingTime="8.020026359s" totalInitContainerRuntime="82.384906ms" isStatefulPod=true podCreationTimestamp="2026-09-10 21:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:15.136754752 +0000 UTC m=+95.156160094" watchObservedRunningTime="2026-09-10 21:00:15.137672505 +0000 UTC m=+95.157077824"
Sep 10 21:00:15.198620 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:15.198001    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-rhkk4" podStartSLOduration=9.613204199 podStartE2EDuration="13.197981049s" totalImagesPullingTime="3.58477685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:15.196623917 +0000 UTC m=+95.216029238" watchObservedRunningTime="2026-09-10 21:00:15.197981049 +0000 UTC m=+95.217386383"
Sep 10 21:00:16.092110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:16.091625    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd" event={"ID":"d65230cd-1a5d-44e2-8d16-216d529dc190","Type":"ContainerStarted","Data":"06d34c395df449901da8779c7e3753151b49672c25ff75c0e517bf09c0514fb9"}
Sep 10 21:00:16.092110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:16.091732    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd" event={"ID":"d65230cd-1a5d-44e2-8d16-216d529dc190","Type":"ContainerStarted","Data":"e82bea5b68b415b0de5d0124c4ac06406e2d55bd5399f21ad0adfa5e8b8f4afb"}
Sep 10 21:00:17.679350 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:17.679258    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69f66476c-vtvls" podUID="cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" containerName="registry" containerID="cri-o://6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4" gracePeriod=30
Sep 10 21:00:17.898137 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:17.898102    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r86bc"
Sep 10 21:00:17.923998 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:17.923949    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78f977ff8f-krggd" podStartSLOduration=5.999122281 podStartE2EDuration="10.923927254s" totalImagesPullingTime="4.924804973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:00:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:00:16.167785122 +0000 UTC m=+96.187190469" watchObservedRunningTime="2026-09-10 21:00:17.923927254 +0000 UTC m=+97.943332572"
Sep 10 21:00:17.969419 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:17.969395    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 21:00:18.101156 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.101121    2557 generic.go:356] "Generic (PLEG): container finished" podID="cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" containerID="6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4" exitCode=0
Sep 10 21:00:18.101156 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.101169    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f66476c-vtvls"
Sep 10 21:00:18.101404 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.101211    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f66476c-vtvls" event={"ID":"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b","Type":"ContainerDied","Data":"6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4"}
Sep 10 21:00:18.101404 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.101262    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f66476c-vtvls" event={"ID":"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b","Type":"ContainerDied","Data":"6535caa41b574304a219f172716c528d2e678c9ed2bb638cca9f93524ee61c25"}
Sep 10 21:00:18.113189 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.113158    2557 scope.go:117] "RemoveContainer" containerID="6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4"
Sep 10 21:00:18.122716 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.122692    2557 scope.go:117] "RemoveContainer" containerID="6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4"
Sep 10 21:00:18.122835 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.122720    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-image-registry-private-configuration\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.122835 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.122774    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-certificates\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.122835 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.122813    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cgrl\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-kube-api-access-2cgrl\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.123001 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.122838    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-installation-pull-secrets\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.123001 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.122873    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-ca-trust-extracted\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.123106 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:00:18.123029    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4\": container with ID starting with 6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4 not found: ID does not exist" containerID="6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4"
Sep 10 21:00:18.123106 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.123067    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4"} err="failed to get container status \"6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4\": rpc error: code = NotFound desc = could not find container \"6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4\": container with ID starting with 6a7e6524a621454773f7edc957fc7396b72e02298480d3636babd556ada96eb4 not found: ID does not exist"
Sep 10 21:00:18.123221 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.123205    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-trusted-ca\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.123283 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.123235    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.123283 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.123271    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-bound-sa-token\") pod \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\" (UID: \"cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b\") "
Sep 10 21:00:18.123545 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.123201    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:18.123545 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.123509    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:00:18.125953 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.125913    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:18.126246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.126220    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-kube-api-access-2cgrl" (OuterVolumeSpecName: "kube-api-access-2cgrl") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "kube-api-access-2cgrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:18.126593 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.126560    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:18.126593 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.126571    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:00:18.127424 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.127396    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:00:18.133970 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.133948    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" (UID: "cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:00:18.224797 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224769    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-trusted-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.224797 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224795    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.224965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224808    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-bound-sa-token\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.224965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224821    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-image-registry-private-configuration\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.224965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224835    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-registry-certificates\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.224965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224881    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cgrl\" (UniqueName: \"kubernetes.io/projected/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-kube-api-access-2cgrl\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.224965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224891    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-installation-pull-secrets\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.224965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.224906    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b-ca-trust-extracted\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:00:18.428438 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.428408    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69f66476c-vtvls"]
Sep 10 21:00:18.436691 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.436652    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69f66476c-vtvls"]
Sep 10 21:00:18.586261 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:18.586186    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" path="/var/lib/kubelet/pods/cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b/volumes"
Sep 10 21:00:47.212265 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:47.212186    2557 generic.go:356] "Generic (PLEG): container finished" podID="aa136b74-710f-492b-9b2b-b24fdc0029a6" containerID="1d134020235318a336a19bc06201cc8831ae6190d313a44620ba4e5869d001c2" exitCode=0
Sep 10 21:00:47.212621 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:47.212259    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s" event={"ID":"aa136b74-710f-492b-9b2b-b24fdc0029a6","Type":"ContainerDied","Data":"1d134020235318a336a19bc06201cc8831ae6190d313a44620ba4e5869d001c2"}
Sep 10 21:00:47.212621 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:47.212553    2557 scope.go:117] "RemoveContainer" containerID="1d134020235318a336a19bc06201cc8831ae6190d313a44620ba4e5869d001c2"
Sep 10 21:00:48.218603 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:00:48.218571    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-sjd8s" event={"ID":"aa136b74-710f-492b-9b2b-b24fdc0029a6","Type":"ContainerStarted","Data":"9f2ea8eecb76750e41e669c45992684b5c36118545644053b1eb2c7061b233a4"}
Sep 10 21:02:05.976951 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.976911    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"]
Sep 10 21:02:05.977543 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.977341    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" containerName="registry"
Sep 10 21:02:05.977543 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.977359    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" containerName="registry"
Sep 10 21:02:05.977543 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.977449    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbc909cd-c0dc-4b77-b0fd-9bcf2473f59b" containerName="registry"
Sep 10 21:02:05.979547 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.979526    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:05.982963 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.982942    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:02:05.983917 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.983901    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p6cbt\""
Sep 10 21:02:05.983999 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.983904    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:02:05.991453 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:05.991433    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"]
Sep 10 21:02:06.075818 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.075777    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr5ft\" (UniqueName: \"kubernetes.io/projected/0b85900e-5a74-43aa-88c0-3e95ecc874a7-kube-api-access-jr5ft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.075980 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.075847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.075980 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.075895    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.176915 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.176884    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.177113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.176933    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr5ft\" (UniqueName: \"kubernetes.io/projected/0b85900e-5a74-43aa-88c0-3e95ecc874a7-kube-api-access-jr5ft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.177113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.176982    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.177339 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.177316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.177402 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.177335    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.186011 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.185987    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr5ft\" (UniqueName: \"kubernetes.io/projected/0b85900e-5a74-43aa-88c0-3e95ecc874a7-kube-api-access-jr5ft\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.288472 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.288395    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:06.561041 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:06.560970    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"]
Sep 10 21:02:06.564437 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:02:06.564409    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b85900e_5a74_43aa_88c0_3e95ecc874a7.slice/crio-fc026bea972ea65137263f6a9ad9e662cf2bd4ba4efcb974fdbf54982aff6dcf WatchSource:0}: Error finding container fc026bea972ea65137263f6a9ad9e662cf2bd4ba4efcb974fdbf54982aff6dcf: Status 404 returned error can't find the container with id fc026bea972ea65137263f6a9ad9e662cf2bd4ba4efcb974fdbf54982aff6dcf
Sep 10 21:02:07.506532 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:07.506491    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc" event={"ID":"0b85900e-5a74-43aa-88c0-3e95ecc874a7","Type":"ContainerStarted","Data":"fc026bea972ea65137263f6a9ad9e662cf2bd4ba4efcb974fdbf54982aff6dcf"}
Sep 10 21:02:16.544198 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:16.544163    2557 generic.go:356] "Generic (PLEG): container finished" podID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerID="4e34e94494d7d37349d80663a0838d47085b245612a263718d28e4bed82f9c64" exitCode=0
Sep 10 21:02:16.544655 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:16.544251    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc" event={"ID":"0b85900e-5a74-43aa-88c0-3e95ecc874a7","Type":"ContainerDied","Data":"4e34e94494d7d37349d80663a0838d47085b245612a263718d28e4bed82f9c64"}
Sep 10 21:02:18.553996 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:18.553964    2557 generic.go:356] "Generic (PLEG): container finished" podID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerID="35d04babf09f71791adb4ab7a3b2918cf765c2e1aad7ac0c4f1165c3f82fda8d" exitCode=0
Sep 10 21:02:18.554292 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:18.554017    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc" event={"ID":"0b85900e-5a74-43aa-88c0-3e95ecc874a7","Type":"ContainerDied","Data":"35d04babf09f71791adb4ab7a3b2918cf765c2e1aad7ac0c4f1165c3f82fda8d"}
Sep 10 21:02:26.591204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:26.591117    2557 generic.go:356] "Generic (PLEG): container finished" podID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerID="c73145b1817703ada7c092f6a1c8e36921912d69085308dffaa8c07a4b3ecf68" exitCode=0
Sep 10 21:02:26.591204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:26.591170    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc" event={"ID":"0b85900e-5a74-43aa-88c0-3e95ecc874a7","Type":"ContainerDied","Data":"c73145b1817703ada7c092f6a1c8e36921912d69085308dffaa8c07a4b3ecf68"}
Sep 10 21:02:27.725484 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.725461    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:27.858652 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.858572    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr5ft\" (UniqueName: \"kubernetes.io/projected/0b85900e-5a74-43aa-88c0-3e95ecc874a7-kube-api-access-jr5ft\") pod \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") "
Sep 10 21:02:27.858652 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.858618    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-util\") pod \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") "
Sep 10 21:02:27.858865 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.858673    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-bundle\") pod \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\" (UID: \"0b85900e-5a74-43aa-88c0-3e95ecc874a7\") "
Sep 10 21:02:27.859387 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.859357    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-bundle" (OuterVolumeSpecName: "bundle") pod "0b85900e-5a74-43aa-88c0-3e95ecc874a7" (UID: "0b85900e-5a74-43aa-88c0-3e95ecc874a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:02:27.860799 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.860764    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b85900e-5a74-43aa-88c0-3e95ecc874a7-kube-api-access-jr5ft" (OuterVolumeSpecName: "kube-api-access-jr5ft") pod "0b85900e-5a74-43aa-88c0-3e95ecc874a7" (UID: "0b85900e-5a74-43aa-88c0-3e95ecc874a7"). InnerVolumeSpecName "kube-api-access-jr5ft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:02:27.863047 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.863021    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-util" (OuterVolumeSpecName: "util") pod "0b85900e-5a74-43aa-88c0-3e95ecc874a7" (UID: "0b85900e-5a74-43aa-88c0-3e95ecc874a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:02:27.959889 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.959858    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:02:27.959889 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.959885    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr5ft\" (UniqueName: \"kubernetes.io/projected/0b85900e-5a74-43aa-88c0-3e95ecc874a7-kube-api-access-jr5ft\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:02:27.960054 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:27.959895    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0b85900e-5a74-43aa-88c0-3e95ecc874a7-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:02:28.599976 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:28.599945    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc" event={"ID":"0b85900e-5a74-43aa-88c0-3e95ecc874a7","Type":"ContainerDied","Data":"fc026bea972ea65137263f6a9ad9e662cf2bd4ba4efcb974fdbf54982aff6dcf"}
Sep 10 21:02:28.600097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:28.599994    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc026bea972ea65137263f6a9ad9e662cf2bd4ba4efcb974fdbf54982aff6dcf"
Sep 10 21:02:28.600097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:28.599955    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rgrdc"
Sep 10 21:02:34.175070 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175035    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"]
Sep 10 21:02:34.175515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175334    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerName="extract"
Sep 10 21:02:34.175515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175345    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerName="extract"
Sep 10 21:02:34.175515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175355    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerName="util"
Sep 10 21:02:34.175515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175360    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerName="util"
Sep 10 21:02:34.175515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175367    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerName="pull"
Sep 10 21:02:34.175515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175372    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerName="pull"
Sep 10 21:02:34.175515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.175420    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b85900e-5a74-43aa-88c0-3e95ecc874a7" containerName="extract"
Sep 10 21:02:34.177379 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.177363    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.180980 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.180950    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-t9l9v\""
Sep 10 21:02:34.181128 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.181022    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:02:34.181128 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.181086    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 21:02:34.194574 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.194551    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"]
Sep 10 21:02:34.206217 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.206197    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31e030a8-4ac5-40d9-a7e2-48ca9616a4df-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-b9h96\" (UID: \"31e030a8-4ac5-40d9-a7e2-48ca9616a4df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.206313 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.206226    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2rhh\" (UniqueName: \"kubernetes.io/projected/31e030a8-4ac5-40d9-a7e2-48ca9616a4df-kube-api-access-q2rhh\") pod \"cert-manager-operator-controller-manager-55c444bf86-b9h96\" (UID: \"31e030a8-4ac5-40d9-a7e2-48ca9616a4df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.307618 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.307562    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31e030a8-4ac5-40d9-a7e2-48ca9616a4df-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-b9h96\" (UID: \"31e030a8-4ac5-40d9-a7e2-48ca9616a4df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.307807 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.307627    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2rhh\" (UniqueName: \"kubernetes.io/projected/31e030a8-4ac5-40d9-a7e2-48ca9616a4df-kube-api-access-q2rhh\") pod \"cert-manager-operator-controller-manager-55c444bf86-b9h96\" (UID: \"31e030a8-4ac5-40d9-a7e2-48ca9616a4df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.307993 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.307972    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31e030a8-4ac5-40d9-a7e2-48ca9616a4df-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-b9h96\" (UID: \"31e030a8-4ac5-40d9-a7e2-48ca9616a4df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.317204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.317169    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2rhh\" (UniqueName: \"kubernetes.io/projected/31e030a8-4ac5-40d9-a7e2-48ca9616a4df-kube-api-access-q2rhh\") pod \"cert-manager-operator-controller-manager-55c444bf86-b9h96\" (UID: \"31e030a8-4ac5-40d9-a7e2-48ca9616a4df\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.486552 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.486524    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"
Sep 10 21:02:34.759790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:34.759762    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96"]
Sep 10 21:02:34.762545 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:02:34.762517    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e030a8_4ac5_40d9_a7e2_48ca9616a4df.slice/crio-0789208108d7a6525568a340337abf1b14fa020a2e9c558c12fa634bdfe30853 WatchSource:0}: Error finding container 0789208108d7a6525568a340337abf1b14fa020a2e9c558c12fa634bdfe30853: Status 404 returned error can't find the container with id 0789208108d7a6525568a340337abf1b14fa020a2e9c558c12fa634bdfe30853
Sep 10 21:02:35.629905 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:35.629871    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96" event={"ID":"31e030a8-4ac5-40d9-a7e2-48ca9616a4df","Type":"ContainerStarted","Data":"0789208108d7a6525568a340337abf1b14fa020a2e9c558c12fa634bdfe30853"}
Sep 10 21:02:36.635829 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:36.635788    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96" event={"ID":"31e030a8-4ac5-40d9-a7e2-48ca9616a4df","Type":"ContainerStarted","Data":"f98a3d1a364012e02204b20a0e86b40c16514e2e7732b9a130b274bc356188c9"}
Sep 10 21:02:36.658935 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:36.658776    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-b9h96" podStartSLOduration=1.216594101 podStartE2EDuration="2.658756531s" totalImagesPullingTime="1.44216243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:02:36.657370505 +0000 UTC m=+236.676775822" watchObservedRunningTime="2026-09-10 21:02:36.658756531 +0000 UTC m=+236.678161849"
Sep 10 21:02:38.072136 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.072102    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"]
Sep 10 21:02:38.074407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.074387    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.077034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.077015    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p6cbt\""
Sep 10 21:02:38.077144 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.077087    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:02:38.077920 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.077900    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:02:38.085063 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.085042    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"]
Sep 10 21:02:38.137832 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.137805    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.137982 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.137836    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.137982 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.137921    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mqjt\" (UniqueName: \"kubernetes.io/projected/6551d1e1-f64f-41d7-a349-8379666ded98-kube-api-access-7mqjt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.239299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.239261    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.239299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.239296    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.239503 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.239356    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mqjt\" (UniqueName: \"kubernetes.io/projected/6551d1e1-f64f-41d7-a349-8379666ded98-kube-api-access-7mqjt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.239768 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.239746    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.239838 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.239762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.251559 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.251528    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mqjt\" (UniqueName: \"kubernetes.io/projected/6551d1e1-f64f-41d7-a349-8379666ded98-kube-api-access-7mqjt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.343788 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.343710    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m6szn"]
Sep 10 21:02:38.345843 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.345827    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.348134 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.348107    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-rllzg\""
Sep 10 21:02:38.348134 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.348132    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 21:02:38.348293 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.348151    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 21:02:38.355876 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.355844    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m6szn"]
Sep 10 21:02:38.382919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.382892    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:38.440362 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.440300    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpk6p\" (UniqueName: \"kubernetes.io/projected/b112e262-7da6-4eac-b871-e63713dcdf98-kube-api-access-zpk6p\") pod \"cert-manager-webhook-6486ffd997-m6szn\" (UID: \"b112e262-7da6-4eac-b871-e63713dcdf98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.440499 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.440467    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b112e262-7da6-4eac-b871-e63713dcdf98-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m6szn\" (UID: \"b112e262-7da6-4eac-b871-e63713dcdf98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.540900 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.540859    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpk6p\" (UniqueName: \"kubernetes.io/projected/b112e262-7da6-4eac-b871-e63713dcdf98-kube-api-access-zpk6p\") pod \"cert-manager-webhook-6486ffd997-m6szn\" (UID: \"b112e262-7da6-4eac-b871-e63713dcdf98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.541091 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.540953    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b112e262-7da6-4eac-b871-e63713dcdf98-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m6szn\" (UID: \"b112e262-7da6-4eac-b871-e63713dcdf98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.549744 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.549713    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b112e262-7da6-4eac-b871-e63713dcdf98-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-m6szn\" (UID: \"b112e262-7da6-4eac-b871-e63713dcdf98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.549850 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.549786    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpk6p\" (UniqueName: \"kubernetes.io/projected/b112e262-7da6-4eac-b871-e63713dcdf98-kube-api-access-zpk6p\") pod \"cert-manager-webhook-6486ffd997-m6szn\" (UID: \"b112e262-7da6-4eac-b871-e63713dcdf98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.650716 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.650667    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"]
Sep 10 21:02:38.652663 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:02:38.652635    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6551d1e1_f64f_41d7_a349_8379666ded98.slice/crio-3ccc93e9e1a0070c06ab7afbd0a7292034b6ff534b9cc002a72bc813ece28fb0 WatchSource:0}: Error finding container 3ccc93e9e1a0070c06ab7afbd0a7292034b6ff534b9cc002a72bc813ece28fb0: Status 404 returned error can't find the container with id 3ccc93e9e1a0070c06ab7afbd0a7292034b6ff534b9cc002a72bc813ece28fb0
Sep 10 21:02:38.666038 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.665792    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:38.939501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:38.939478    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-m6szn"]
Sep 10 21:02:38.941884 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:02:38.941854    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb112e262_7da6_4eac_b871_e63713dcdf98.slice/crio-5a57d5515b32be83df641c579e21f75564da2b9e1ad1724f42e8db6f23317158 WatchSource:0}: Error finding container 5a57d5515b32be83df641c579e21f75564da2b9e1ad1724f42e8db6f23317158: Status 404 returned error can't find the container with id 5a57d5515b32be83df641c579e21f75564da2b9e1ad1724f42e8db6f23317158
Sep 10 21:02:39.649880 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:39.649837    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn" event={"ID":"b112e262-7da6-4eac-b871-e63713dcdf98","Type":"ContainerStarted","Data":"5a57d5515b32be83df641c579e21f75564da2b9e1ad1724f42e8db6f23317158"}
Sep 10 21:02:39.651611 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:39.651579    2557 generic.go:356] "Generic (PLEG): container finished" podID="6551d1e1-f64f-41d7-a349-8379666ded98" containerID="773f303253f70d94b700d2263971037515d4d39af0b2d87d42cff360f1bca270" exitCode=0
Sep 10 21:02:39.651762 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:39.651658    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh" event={"ID":"6551d1e1-f64f-41d7-a349-8379666ded98","Type":"ContainerDied","Data":"773f303253f70d94b700d2263971037515d4d39af0b2d87d42cff360f1bca270"}
Sep 10 21:02:39.651812 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:39.651766    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh" event={"ID":"6551d1e1-f64f-41d7-a349-8379666ded98","Type":"ContainerStarted","Data":"3ccc93e9e1a0070c06ab7afbd0a7292034b6ff534b9cc002a72bc813ece28fb0"}
Sep 10 21:02:42.673394 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:42.673294    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn" event={"ID":"b112e262-7da6-4eac-b871-e63713dcdf98","Type":"ContainerStarted","Data":"9b7e18c6e0f3b1ad32fc41bf35fdd9e536df2ddbad1bdd7bd971d2dd9626a276"}
Sep 10 21:02:42.673394 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:42.673367    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:42.675238 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:42.675216    2557 generic.go:356] "Generic (PLEG): container finished" podID="6551d1e1-f64f-41d7-a349-8379666ded98" containerID="e67968ea02e5222b1e0307959f249232c43a75e81ba881766eafff81f1d06d8f" exitCode=0
Sep 10 21:02:42.675334 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:42.675274    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh" event={"ID":"6551d1e1-f64f-41d7-a349-8379666ded98","Type":"ContainerDied","Data":"e67968ea02e5222b1e0307959f249232c43a75e81ba881766eafff81f1d06d8f"}
Sep 10 21:02:42.692939 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:42.692903    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn" podStartSLOduration=1.386296073 podStartE2EDuration="4.692889574s" totalImagesPullingTime="3.306593501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:02:42.691466577 +0000 UTC m=+242.710871895" watchObservedRunningTime="2026-09-10 21:02:42.692889574 +0000 UTC m=+242.712294892"
Sep 10 21:02:43.682637 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:43.682599    2557 generic.go:356] "Generic (PLEG): container finished" podID="6551d1e1-f64f-41d7-a349-8379666ded98" containerID="a470ceca4a2090baea6a703a73cc1993a9a07d046e0c0d31769e5a09f1368fb3" exitCode=0
Sep 10 21:02:43.683050 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:43.682697    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh" event={"ID":"6551d1e1-f64f-41d7-a349-8379666ded98","Type":"ContainerDied","Data":"a470ceca4a2090baea6a703a73cc1993a9a07d046e0c0d31769e5a09f1368fb3"}
Sep 10 21:02:44.807033 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.807002    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:44.895175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.895146    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-bundle\") pod \"6551d1e1-f64f-41d7-a349-8379666ded98\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") "
Sep 10 21:02:44.895337 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.895192    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mqjt\" (UniqueName: \"kubernetes.io/projected/6551d1e1-f64f-41d7-a349-8379666ded98-kube-api-access-7mqjt\") pod \"6551d1e1-f64f-41d7-a349-8379666ded98\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") "
Sep 10 21:02:44.895337 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.895245    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-util\") pod \"6551d1e1-f64f-41d7-a349-8379666ded98\" (UID: \"6551d1e1-f64f-41d7-a349-8379666ded98\") "
Sep 10 21:02:44.895573 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.895541    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-bundle" (OuterVolumeSpecName: "bundle") pod "6551d1e1-f64f-41d7-a349-8379666ded98" (UID: "6551d1e1-f64f-41d7-a349-8379666ded98"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:02:44.897457 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.897429    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6551d1e1-f64f-41d7-a349-8379666ded98-kube-api-access-7mqjt" (OuterVolumeSpecName: "kube-api-access-7mqjt") pod "6551d1e1-f64f-41d7-a349-8379666ded98" (UID: "6551d1e1-f64f-41d7-a349-8379666ded98"). InnerVolumeSpecName "kube-api-access-7mqjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:02:44.901092 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.901056    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-util" (OuterVolumeSpecName: "util") pod "6551d1e1-f64f-41d7-a349-8379666ded98" (UID: "6551d1e1-f64f-41d7-a349-8379666ded98"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:02:44.996664 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.996638    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:02:44.996664 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.996662    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6551d1e1-f64f-41d7-a349-8379666ded98-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:02:44.996828 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:44.996671    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mqjt\" (UniqueName: \"kubernetes.io/projected/6551d1e1-f64f-41d7-a349-8379666ded98-kube-api-access-7mqjt\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:02:45.693624 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:45.693594    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh"
Sep 10 21:02:45.693624 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:45.693615    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fsb5jh" event={"ID":"6551d1e1-f64f-41d7-a349-8379666ded98","Type":"ContainerDied","Data":"3ccc93e9e1a0070c06ab7afbd0a7292034b6ff534b9cc002a72bc813ece28fb0"}
Sep 10 21:02:45.693834 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:45.693656    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ccc93e9e1a0070c06ab7afbd0a7292034b6ff534b9cc002a72bc813ece28fb0"
Sep 10 21:02:48.684816 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:48.684786    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-m6szn"
Sep 10 21:02:56.391506 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391466    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"]
Sep 10 21:02:56.391919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391844    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6551d1e1-f64f-41d7-a349-8379666ded98" containerName="pull"
Sep 10 21:02:56.391919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391856    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6551d1e1-f64f-41d7-a349-8379666ded98" containerName="pull"
Sep 10 21:02:56.391919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391866    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6551d1e1-f64f-41d7-a349-8379666ded98" containerName="util"
Sep 10 21:02:56.391919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391871    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6551d1e1-f64f-41d7-a349-8379666ded98" containerName="util"
Sep 10 21:02:56.391919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391882    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6551d1e1-f64f-41d7-a349-8379666ded98" containerName="extract"
Sep 10 21:02:56.391919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391888    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6551d1e1-f64f-41d7-a349-8379666ded98" containerName="extract"
Sep 10 21:02:56.392112 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.391944    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6551d1e1-f64f-41d7-a349-8379666ded98" containerName="extract"
Sep 10 21:02:56.417820 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.417775    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"]
Sep 10 21:02:56.417972 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.417862    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:56.420667 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.420645    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:02:56.420775 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.420762    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vfk4m\""
Sep 10 21:02:56.421641 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.421625    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 21:02:56.483375 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.483343    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx85g\" (UniqueName: \"kubernetes.io/projected/10c49916-fcf0-4c69-b821-9dcf4e2f4899-kube-api-access-xx85g\") pod \"openshift-lws-operator-fd8ccff4c-k7g45\" (UID: \"10c49916-fcf0-4c69-b821-9dcf4e2f4899\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:56.483533 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.483389    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10c49916-fcf0-4c69-b821-9dcf4e2f4899-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k7g45\" (UID: \"10c49916-fcf0-4c69-b821-9dcf4e2f4899\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:56.584014 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.583979    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx85g\" (UniqueName: \"kubernetes.io/projected/10c49916-fcf0-4c69-b821-9dcf4e2f4899-kube-api-access-xx85g\") pod \"openshift-lws-operator-fd8ccff4c-k7g45\" (UID: \"10c49916-fcf0-4c69-b821-9dcf4e2f4899\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:56.584176 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.584042    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10c49916-fcf0-4c69-b821-9dcf4e2f4899-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k7g45\" (UID: \"10c49916-fcf0-4c69-b821-9dcf4e2f4899\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:56.584476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.584450    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10c49916-fcf0-4c69-b821-9dcf4e2f4899-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k7g45\" (UID: \"10c49916-fcf0-4c69-b821-9dcf4e2f4899\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:56.593387 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.593356    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx85g\" (UniqueName: \"kubernetes.io/projected/10c49916-fcf0-4c69-b821-9dcf4e2f4899-kube-api-access-xx85g\") pod \"openshift-lws-operator-fd8ccff4c-k7g45\" (UID: \"10c49916-fcf0-4c69-b821-9dcf4e2f4899\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:56.727152 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:56.727116    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"
Sep 10 21:02:57.013843 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:57.013816    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45"]
Sep 10 21:02:57.016116 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:02:57.016089    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10c49916_fcf0_4c69_b821_9dcf4e2f4899.slice/crio-c4bdca28621b3cb3374459fe75fa19007c51c699d6a5e5dddfa44b48d9fe07ab WatchSource:0}: Error finding container c4bdca28621b3cb3374459fe75fa19007c51c699d6a5e5dddfa44b48d9fe07ab: Status 404 returned error can't find the container with id c4bdca28621b3cb3374459fe75fa19007c51c699d6a5e5dddfa44b48d9fe07ab
Sep 10 21:02:57.747710 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:57.747661    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45" event={"ID":"10c49916-fcf0-4c69-b821-9dcf4e2f4899","Type":"ContainerStarted","Data":"c4bdca28621b3cb3374459fe75fa19007c51c699d6a5e5dddfa44b48d9fe07ab"}
Sep 10 21:02:58.754019 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:58.753985    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45" event={"ID":"10c49916-fcf0-4c69-b821-9dcf4e2f4899","Type":"ContainerStarted","Data":"33911e5cc16d5c00cb04d2f3530204513907156925d75f240ccbb5c9ef6dd98b"}
Sep 10 21:02:58.771545 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:02:58.771488    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k7g45" podStartSLOduration=1.228850104 podStartE2EDuration="2.771470574s" totalImagesPullingTime="1.54262047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:02:58.770782203 +0000 UTC m=+258.790187522" watchObservedRunningTime="2026-09-10 21:02:58.771470574 +0000 UTC m=+258.790875891"
Sep 10 21:03:01.628009 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.627976    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"]
Sep 10 21:03:01.630562 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.630545    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.632897 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.632873    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:03:01.633777 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.633761    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p6cbt\""
Sep 10 21:03:01.633873 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.633763    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:03:01.640919 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.640895    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"]
Sep 10 21:03:01.728345 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.728311    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxk7n\" (UniqueName: \"kubernetes.io/projected/8cb93b8f-8904-420e-a8a1-8ab807c2a478-kube-api-access-cxk7n\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.728501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.728360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.728501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.728427    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.828983 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.828952    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxk7n\" (UniqueName: \"kubernetes.io/projected/8cb93b8f-8904-420e-a8a1-8ab807c2a478-kube-api-access-cxk7n\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.829148 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.829011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.829148 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.829048    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.829419 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.829398    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.829490 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.829472    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.838118 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.838097    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxk7n\" (UniqueName: \"kubernetes.io/projected/8cb93b8f-8904-420e-a8a1-8ab807c2a478-kube-api-access-cxk7n\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:01.940102 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:01.940017    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:02.212009 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:02.211985    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"]
Sep 10 21:03:02.213809 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:03:02.213783    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb93b8f_8904_420e_a8a1_8ab807c2a478.slice/crio-89377dad5742d11cbc34c0f3cd9a6fab82b817a7fe5e97f7eabf74097d93bc03 WatchSource:0}: Error finding container 89377dad5742d11cbc34c0f3cd9a6fab82b817a7fe5e97f7eabf74097d93bc03: Status 404 returned error can't find the container with id 89377dad5742d11cbc34c0f3cd9a6fab82b817a7fe5e97f7eabf74097d93bc03
Sep 10 21:03:02.773123 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:02.773087    2557 generic.go:356] "Generic (PLEG): container finished" podID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerID="ec53c64d74cdf862e3b1522518ccdf69455c56ccab8c086cc6e812219d3e9442" exitCode=0
Sep 10 21:03:02.773503 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:02.773179    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4" event={"ID":"8cb93b8f-8904-420e-a8a1-8ab807c2a478","Type":"ContainerDied","Data":"ec53c64d74cdf862e3b1522518ccdf69455c56ccab8c086cc6e812219d3e9442"}
Sep 10 21:03:02.773503 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:02.773236    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4" event={"ID":"8cb93b8f-8904-420e-a8a1-8ab807c2a478","Type":"ContainerStarted","Data":"89377dad5742d11cbc34c0f3cd9a6fab82b817a7fe5e97f7eabf74097d93bc03"}
Sep 10 21:03:03.779535 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:03.779497    2557 generic.go:356] "Generic (PLEG): container finished" podID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerID="fe689ec04d0653d6cb425bc353ca72ec9fb298a28785445afedd94912ed8e9b5" exitCode=0
Sep 10 21:03:03.780024 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:03.779563    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4" event={"ID":"8cb93b8f-8904-420e-a8a1-8ab807c2a478","Type":"ContainerDied","Data":"fe689ec04d0653d6cb425bc353ca72ec9fb298a28785445afedd94912ed8e9b5"}
Sep 10 21:03:04.786200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:04.786168    2557 generic.go:356] "Generic (PLEG): container finished" podID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerID="f1450bbf56246e5b3ac21a76d148c57e43a60c48de7b2b0eaba6eb071338de87" exitCode=0
Sep 10 21:03:04.786579 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:04.786240    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4" event={"ID":"8cb93b8f-8904-420e-a8a1-8ab807c2a478","Type":"ContainerDied","Data":"f1450bbf56246e5b3ac21a76d148c57e43a60c48de7b2b0eaba6eb071338de87"}
Sep 10 21:03:05.913055 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:05.913023    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:06.067093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.067002    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-util\") pod \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") "
Sep 10 21:03:06.067093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.067049    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxk7n\" (UniqueName: \"kubernetes.io/projected/8cb93b8f-8904-420e-a8a1-8ab807c2a478-kube-api-access-cxk7n\") pod \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") "
Sep 10 21:03:06.067093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.067067    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-bundle\") pod \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\" (UID: \"8cb93b8f-8904-420e-a8a1-8ab807c2a478\") "
Sep 10 21:03:06.067870 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.067843    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-bundle" (OuterVolumeSpecName: "bundle") pod "8cb93b8f-8904-420e-a8a1-8ab807c2a478" (UID: "8cb93b8f-8904-420e-a8a1-8ab807c2a478"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:03:06.069278 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.069252    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb93b8f-8904-420e-a8a1-8ab807c2a478-kube-api-access-cxk7n" (OuterVolumeSpecName: "kube-api-access-cxk7n") pod "8cb93b8f-8904-420e-a8a1-8ab807c2a478" (UID: "8cb93b8f-8904-420e-a8a1-8ab807c2a478"). InnerVolumeSpecName "kube-api-access-cxk7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:03:06.072511 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.072489    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-util" (OuterVolumeSpecName: "util") pod "8cb93b8f-8904-420e-a8a1-8ab807c2a478" (UID: "8cb93b8f-8904-420e-a8a1-8ab807c2a478"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:03:06.167991 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.167960    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:03:06.167991 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.167986    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxk7n\" (UniqueName: \"kubernetes.io/projected/8cb93b8f-8904-420e-a8a1-8ab807c2a478-kube-api-access-cxk7n\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:03:06.167991 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.167997    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb93b8f-8904-420e-a8a1-8ab807c2a478-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:03:06.796392 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.796363    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4"
Sep 10 21:03:06.796392 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.796375    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2sbw4" event={"ID":"8cb93b8f-8904-420e-a8a1-8ab807c2a478","Type":"ContainerDied","Data":"89377dad5742d11cbc34c0f3cd9a6fab82b817a7fe5e97f7eabf74097d93bc03"}
Sep 10 21:03:06.796629 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:06.796413    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89377dad5742d11cbc34c0f3cd9a6fab82b817a7fe5e97f7eabf74097d93bc03"
Sep 10 21:03:17.043526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.043493    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"]
Sep 10 21:03:17.044100 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.043993    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerName="extract"
Sep 10 21:03:17.044100 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.044013    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerName="extract"
Sep 10 21:03:17.044100 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.044044    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerName="pull"
Sep 10 21:03:17.044100 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.044053    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerName="pull"
Sep 10 21:03:17.044100 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.044067    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerName="util"
Sep 10 21:03:17.044100 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.044078    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerName="util"
Sep 10 21:03:17.044442 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.044167    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb93b8f-8904-420e-a8a1-8ab807c2a478" containerName="extract"
Sep 10 21:03:17.047121 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.047100    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.051407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.051385    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p6cbt\""
Sep 10 21:03:17.051493 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.051408    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:03:17.052473 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.052445    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:03:17.056855 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.056828    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8pd8\" (UniqueName: \"kubernetes.io/projected/cb320389-54d5-42b9-9264-45f6bec21380-kube-api-access-q8pd8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.056967 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.056863    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.056967 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.056894    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.056967 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.056935    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"]
Sep 10 21:03:17.158026 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.157981    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.158211 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.158120    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8pd8\" (UniqueName: \"kubernetes.io/projected/cb320389-54d5-42b9-9264-45f6bec21380-kube-api-access-q8pd8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.158211 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.158159    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.158410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.158390    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.158488 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.158472    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.170462 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.170439    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8pd8\" (UniqueName: \"kubernetes.io/projected/cb320389-54d5-42b9-9264-45f6bec21380-kube-api-access-q8pd8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.356657 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.356582    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:17.640989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.640962    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"]
Sep 10 21:03:17.642572 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:03:17.642540    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb320389_54d5_42b9_9264_45f6bec21380.slice/crio-8b501a8b7c8b6aaf2a41b1e8c6b915347f2f11e05046f0b1563254cbf1979626 WatchSource:0}: Error finding container 8b501a8b7c8b6aaf2a41b1e8c6b915347f2f11e05046f0b1563254cbf1979626: Status 404 returned error can't find the container with id 8b501a8b7c8b6aaf2a41b1e8c6b915347f2f11e05046f0b1563254cbf1979626
Sep 10 21:03:17.850753 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.850721    2557 generic.go:356] "Generic (PLEG): container finished" podID="cb320389-54d5-42b9-9264-45f6bec21380" containerID="6919232e165b132fff8779ca8bead6fdb9722e55743affbee5e3fcb8107e5fbc" exitCode=0
Sep 10 21:03:17.851063 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.850789    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g" event={"ID":"cb320389-54d5-42b9-9264-45f6bec21380","Type":"ContainerDied","Data":"6919232e165b132fff8779ca8bead6fdb9722e55743affbee5e3fcb8107e5fbc"}
Sep 10 21:03:17.851063 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:17.850854    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g" event={"ID":"cb320389-54d5-42b9-9264-45f6bec21380","Type":"ContainerStarted","Data":"8b501a8b7c8b6aaf2a41b1e8c6b915347f2f11e05046f0b1563254cbf1979626"}
Sep 10 21:03:18.197911 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.197880    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"]
Sep 10 21:03:18.200141 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.200124    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.204554 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.204531    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 21:03:18.204732 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.204557    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 21:03:18.204791 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.204737    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 21:03:18.204830 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.204802    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 21:03:18.204830 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.204819    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ldf8n\""
Sep 10 21:03:18.221098 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.221075    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"]
Sep 10 21:03:18.266040 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.266010    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d8de411-a529-4246-9c4c-309d9bdb4e07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.266205 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.266045    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld62s\" (UniqueName: \"kubernetes.io/projected/1d8de411-a529-4246-9c4c-309d9bdb4e07-kube-api-access-ld62s\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.266278 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.266199    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8de411-a529-4246-9c4c-309d9bdb4e07-tmp\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.266278 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.266241    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d8de411-a529-4246-9c4c-309d9bdb4e07-webhook-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.266372 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.266311    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d8de411-a529-4246-9c4c-309d9bdb4e07-manifests\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.367182 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.367150    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8de411-a529-4246-9c4c-309d9bdb4e07-tmp\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.367303 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.367185    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d8de411-a529-4246-9c4c-309d9bdb4e07-webhook-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.367303 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.367214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d8de411-a529-4246-9c4c-309d9bdb4e07-manifests\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.367303 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.367250    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d8de411-a529-4246-9c4c-309d9bdb4e07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.367303 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.367268    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld62s\" (UniqueName: \"kubernetes.io/projected/1d8de411-a529-4246-9c4c-309d9bdb4e07-kube-api-access-ld62s\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.367631 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.367611    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1d8de411-a529-4246-9c4c-309d9bdb4e07-manifests\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.369399 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.369372    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d8de411-a529-4246-9c4c-309d9bdb4e07-tmp\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.369508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.369489    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1d8de411-a529-4246-9c4c-309d9bdb4e07-webhook-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.369703 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.369661    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1d8de411-a529-4246-9c4c-309d9bdb4e07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.382825 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.382801    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld62s\" (UniqueName: \"kubernetes.io/projected/1d8de411-a529-4246-9c4c-309d9bdb4e07-kube-api-access-ld62s\") pod \"opendatahub-operator-controller-manager-55d69f6bb5-2xnnx\" (UID: \"1d8de411-a529-4246-9c4c-309d9bdb4e07\") " pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.510102 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.510074    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:18.792358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.792329    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"]
Sep 10 21:03:18.793365 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:03:18.793337    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d8de411_a529_4246_9c4c_309d9bdb4e07.slice/crio-b1623c8519dcbd792448697c28fe453a2d467866bb9a4b1643b03b7d9c09c458 WatchSource:0}: Error finding container b1623c8519dcbd792448697c28fe453a2d467866bb9a4b1643b03b7d9c09c458: Status 404 returned error can't find the container with id b1623c8519dcbd792448697c28fe453a2d467866bb9a4b1643b03b7d9c09c458
Sep 10 21:03:18.856443 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.856408    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx" event={"ID":"1d8de411-a529-4246-9c4c-309d9bdb4e07","Type":"ContainerStarted","Data":"b1623c8519dcbd792448697c28fe453a2d467866bb9a4b1643b03b7d9c09c458"}
Sep 10 21:03:18.858136 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.858109    2557 generic.go:356] "Generic (PLEG): container finished" podID="cb320389-54d5-42b9-9264-45f6bec21380" containerID="3a191a5a504fc0af2d6934f6d17ab18705da317d7b6ee28aa711aada3419999c" exitCode=0
Sep 10 21:03:18.858249 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:18.858194    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g" event={"ID":"cb320389-54d5-42b9-9264-45f6bec21380","Type":"ContainerDied","Data":"3a191a5a504fc0af2d6934f6d17ab18705da317d7b6ee28aa711aada3419999c"}
Sep 10 21:03:19.867228 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:19.867103    2557 generic.go:356] "Generic (PLEG): container finished" podID="cb320389-54d5-42b9-9264-45f6bec21380" containerID="1b00d05119c38583a761978ac1fd8ab1fff98f5ff830b1c467466779dc9aa1ea" exitCode=0
Sep 10 21:03:19.867228 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:19.867140    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g" event={"ID":"cb320389-54d5-42b9-9264-45f6bec21380","Type":"ContainerDied","Data":"1b00d05119c38583a761978ac1fd8ab1fff98f5ff830b1c467466779dc9aa1ea"}
Sep 10 21:03:21.629701 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.629656    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:21.692834 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.692811    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-util\") pod \"cb320389-54d5-42b9-9264-45f6bec21380\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") "
Sep 10 21:03:21.692948 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.692855    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-bundle\") pod \"cb320389-54d5-42b9-9264-45f6bec21380\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") "
Sep 10 21:03:21.692948 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.692915    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8pd8\" (UniqueName: \"kubernetes.io/projected/cb320389-54d5-42b9-9264-45f6bec21380-kube-api-access-q8pd8\") pod \"cb320389-54d5-42b9-9264-45f6bec21380\" (UID: \"cb320389-54d5-42b9-9264-45f6bec21380\") "
Sep 10 21:03:21.693697 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.693656    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-bundle" (OuterVolumeSpecName: "bundle") pod "cb320389-54d5-42b9-9264-45f6bec21380" (UID: "cb320389-54d5-42b9-9264-45f6bec21380"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:03:21.694848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.694818    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb320389-54d5-42b9-9264-45f6bec21380-kube-api-access-q8pd8" (OuterVolumeSpecName: "kube-api-access-q8pd8") pod "cb320389-54d5-42b9-9264-45f6bec21380" (UID: "cb320389-54d5-42b9-9264-45f6bec21380"). InnerVolumeSpecName "kube-api-access-q8pd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:03:21.700416 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.700373    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-util" (OuterVolumeSpecName: "util") pod "cb320389-54d5-42b9-9264-45f6bec21380" (UID: "cb320389-54d5-42b9-9264-45f6bec21380"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:03:21.793839 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.793806    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:03:21.793839 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.793832    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb320389-54d5-42b9-9264-45f6bec21380-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:03:21.793839 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.793841    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8pd8\" (UniqueName: \"kubernetes.io/projected/cb320389-54d5-42b9-9264-45f6bec21380-kube-api-access-q8pd8\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:03:21.879578 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.879500    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx" event={"ID":"1d8de411-a529-4246-9c4c-309d9bdb4e07","Type":"ContainerStarted","Data":"56223e52935efc4a3e3cbc1bc3dc3386455c4a1b11381f342c4373d7e7566cde"}
Sep 10 21:03:21.881409 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.881386    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g" event={"ID":"cb320389-54d5-42b9-9264-45f6bec21380","Type":"ContainerDied","Data":"8b501a8b7c8b6aaf2a41b1e8c6b915347f2f11e05046f0b1563254cbf1979626"}
Sep 10 21:03:21.881508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.881420    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b501a8b7c8b6aaf2a41b1e8c6b915347f2f11e05046f0b1563254cbf1979626"
Sep 10 21:03:21.881508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:21.881430    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0rg88g"
Sep 10 21:03:22.887432 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:22.887395    2557 generic.go:356] "Generic (PLEG): container finished" podID="1d8de411-a529-4246-9c4c-309d9bdb4e07" containerID="56223e52935efc4a3e3cbc1bc3dc3386455c4a1b11381f342c4373d7e7566cde" exitCode=0
Sep 10 21:03:22.887790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:22.887447    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx" event={"ID":"1d8de411-a529-4246-9c4c-309d9bdb4e07","Type":"ContainerDied","Data":"56223e52935efc4a3e3cbc1bc3dc3386455c4a1b11381f342c4373d7e7566cde"}
Sep 10 21:03:23.895776 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:23.895735    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx" event={"ID":"1d8de411-a529-4246-9c4c-309d9bdb4e07","Type":"ContainerStarted","Data":"2a19a012e0622f2b5c61132a5e164f87716b0fd75a4786a9cf0e7705af6e272a"}
Sep 10 21:03:23.896157 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:23.895874    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:23.945015 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:23.944956    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx" podStartSLOduration=1.689162664 podStartE2EDuration="5.944934391s" totalImagesPullingTime="3.189041467s" totalInitContainerRuntime="1.06673026s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:23.943203054 +0000 UTC m=+283.962608373" watchObservedRunningTime="2026-09-10 21:03:23.944934391 +0000 UTC m=+283.964339711"
Sep 10 21:03:34.903115 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:34.903079    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55d69f6bb5-2xnnx"
Sep 10 21:03:35.429652 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.429611    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"]
Sep 10 21:03:35.429976 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.429962    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb320389-54d5-42b9-9264-45f6bec21380" containerName="extract"
Sep 10 21:03:35.429976 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.429977    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb320389-54d5-42b9-9264-45f6bec21380" containerName="extract"
Sep 10 21:03:35.430064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.429986    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb320389-54d5-42b9-9264-45f6bec21380" containerName="util"
Sep 10 21:03:35.430064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.429991    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb320389-54d5-42b9-9264-45f6bec21380" containerName="util"
Sep 10 21:03:35.430064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.430013    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb320389-54d5-42b9-9264-45f6bec21380" containerName="pull"
Sep 10 21:03:35.430064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.430018    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb320389-54d5-42b9-9264-45f6bec21380" containerName="pull"
Sep 10 21:03:35.430254 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.430070    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb320389-54d5-42b9-9264-45f6bec21380" containerName="extract"
Sep 10 21:03:35.432512 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.432490    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.436005 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.435984    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c4kh2\""
Sep 10 21:03:35.436107 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.436004    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 21:03:35.436107 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.436030    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 21:03:35.436107 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.436030    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 21:03:35.445343 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.445312    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"]
Sep 10 21:03:35.484383 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.484359    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/814d335d-d557-46cd-ba4c-ace8e839d7b7-metrics-cert\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.484486 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.484402    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/814d335d-d557-46cd-ba4c-ace8e839d7b7-manager-config\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.484486 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.484425    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4gnr\" (UniqueName: \"kubernetes.io/projected/814d335d-d557-46cd-ba4c-ace8e839d7b7-kube-api-access-m4gnr\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.484569 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.484493    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/814d335d-d557-46cd-ba4c-ace8e839d7b7-cert\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.584926 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.584896    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/814d335d-d557-46cd-ba4c-ace8e839d7b7-cert\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.585058 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.584945    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/814d335d-d557-46cd-ba4c-ace8e839d7b7-metrics-cert\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.585058 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.584973    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/814d335d-d557-46cd-ba4c-ace8e839d7b7-manager-config\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.585172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.585132    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4gnr\" (UniqueName: \"kubernetes.io/projected/814d335d-d557-46cd-ba4c-ace8e839d7b7-kube-api-access-m4gnr\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.585665 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.585644    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/814d335d-d557-46cd-ba4c-ace8e839d7b7-manager-config\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.587347 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.587328    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/814d335d-d557-46cd-ba4c-ace8e839d7b7-metrics-cert\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.587447 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.587414    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/814d335d-d557-46cd-ba4c-ace8e839d7b7-cert\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.595897 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.595874    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4gnr\" (UniqueName: \"kubernetes.io/projected/814d335d-d557-46cd-ba4c-ace8e839d7b7-kube-api-access-m4gnr\") pod \"lws-controller-manager-79fcf6dff7-6dm5x\" (UID: \"814d335d-d557-46cd-ba4c-ace8e839d7b7\") " pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:35.742139 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:35.742116    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:36.014794 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:36.014718    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"]
Sep 10 21:03:36.021033 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:03:36.021008    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814d335d_d557_46cd_ba4c_ace8e839d7b7.slice/crio-bc5ac027f282a7923d1c24599559e09bab1a83efbc5c6412287e28f62deed8d6 WatchSource:0}: Error finding container bc5ac027f282a7923d1c24599559e09bab1a83efbc5c6412287e28f62deed8d6: Status 404 returned error can't find the container with id bc5ac027f282a7923d1c24599559e09bab1a83efbc5c6412287e28f62deed8d6
Sep 10 21:03:36.958972 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:36.958901    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x" event={"ID":"814d335d-d557-46cd-ba4c-ace8e839d7b7","Type":"ContainerStarted","Data":"bc5ac027f282a7923d1c24599559e09bab1a83efbc5c6412287e28f62deed8d6"}
Sep 10 21:03:37.968099 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:37.968058    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x" event={"ID":"814d335d-d557-46cd-ba4c-ace8e839d7b7","Type":"ContainerStarted","Data":"ed3aa14f6e50a4bc649c0e03be74b137cb828f005014d64e24bd5466ee0c4d93"}
Sep 10 21:03:37.968470 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:37.968129    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:03:38.089027 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.088976    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x" podStartSLOduration=1.724591668 podStartE2EDuration="3.088955453s" totalImagesPullingTime="1.364363785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:38.087250512 +0000 UTC m=+298.106655830" watchObservedRunningTime="2026-09-10 21:03:38.088955453 +0000 UTC m=+298.108360773"
Sep 10 21:03:38.518549 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.518515    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"]
Sep 10 21:03:38.521021 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.520997    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.524430 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.524402    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 21:03:38.524576 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.524547    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 21:03:38.524576 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.524559    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 21:03:38.524712 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.524670    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5g5rd\""
Sep 10 21:03:38.525089 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.525073    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 21:03:38.538416 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.538384    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"]
Sep 10 21:03:38.611999 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.611964    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7jnc\" (UniqueName: \"kubernetes.io/projected/ae9a0726-e680-4910-a364-c5f46d94210b-kube-api-access-w7jnc\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.612175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.612015    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae9a0726-e680-4910-a364-c5f46d94210b-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.612175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.612091    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae9a0726-e680-4910-a364-c5f46d94210b-tmp\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.712829 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.712792    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7jnc\" (UniqueName: \"kubernetes.io/projected/ae9a0726-e680-4910-a364-c5f46d94210b-kube-api-access-w7jnc\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.713012 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.712850    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae9a0726-e680-4910-a364-c5f46d94210b-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.713012 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.712881    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae9a0726-e680-4910-a364-c5f46d94210b-tmp\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.713177 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:03:38.713007    2557 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 21:03:38.713177 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:03:38.713071    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae9a0726-e680-4910-a364-c5f46d94210b-tls-certs podName:ae9a0726-e680-4910-a364-c5f46d94210b nodeName:}" failed. No retries permitted until 2026-09-10 21:03:39.213053116 +0000 UTC m=+299.232458411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ae9a0726-e680-4910-a364-c5f46d94210b-tls-certs") pod "kube-auth-proxy-cd8d54847-5tts4" (UID: "ae9a0726-e680-4910-a364-c5f46d94210b") : secret "kube-auth-proxy-tls" not found
Sep 10 21:03:38.715257 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.715225    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae9a0726-e680-4910-a364-c5f46d94210b-tmp\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:38.726635 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:38.726615    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7jnc\" (UniqueName: \"kubernetes.io/projected/ae9a0726-e680-4910-a364-c5f46d94210b-kube-api-access-w7jnc\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:39.216876 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:39.216838    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae9a0726-e680-4910-a364-c5f46d94210b-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:39.219246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:39.219218    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae9a0726-e680-4910-a364-c5f46d94210b-tls-certs\") pod \"kube-auth-proxy-cd8d54847-5tts4\" (UID: \"ae9a0726-e680-4910-a364-c5f46d94210b\") " pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:39.433043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:39.433008    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"
Sep 10 21:03:39.726324 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:39.726297    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-cd8d54847-5tts4"]
Sep 10 21:03:39.726796 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:03:39.726768    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae9a0726_e680_4910_a364_c5f46d94210b.slice/crio-1c94cc4d977e4835950ab8fbcbf10a852d1c386f7fb4c7021f8b2dd898546ee2 WatchSource:0}: Error finding container 1c94cc4d977e4835950ab8fbcbf10a852d1c386f7fb4c7021f8b2dd898546ee2: Status 404 returned error can't find the container with id 1c94cc4d977e4835950ab8fbcbf10a852d1c386f7fb4c7021f8b2dd898546ee2
Sep 10 21:03:39.985317 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:39.985229    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4" event={"ID":"ae9a0726-e680-4910-a364-c5f46d94210b","Type":"ContainerStarted","Data":"1c94cc4d977e4835950ab8fbcbf10a852d1c386f7fb4c7021f8b2dd898546ee2"}
Sep 10 21:03:40.354555 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:40.354480    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:03:40.357502 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:40.357476    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:03:40.364719 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:40.364695    2557 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 21:03:42.462998 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:42.462970    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 21:03:43.001148 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:43.001110    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4" event={"ID":"ae9a0726-e680-4910-a364-c5f46d94210b","Type":"ContainerStarted","Data":"778732650b7472f010fa84c78e9e16c295bb3a9824b6fa7f14bff7b9c3c0e4f1"}
Sep 10 21:03:43.018802 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:43.018751    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-cd8d54847-5tts4" podStartSLOduration=2.287780531 podStartE2EDuration="5.018735314s" totalImagesPullingTime="2.730954783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:03:43.017393091 +0000 UTC m=+303.036798406" watchObservedRunningTime="2026-09-10 21:03:43.018735314 +0000 UTC m=+303.038140632"
Sep 10 21:03:48.981812 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:03:48.981778    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-79fcf6dff7-6dm5x"
Sep 10 21:04:53.550633 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.550553    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"]
Sep 10 21:04:53.552992 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.552976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.558275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.558255    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-p6cbt\""
Sep 10 21:04:53.558596 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.558579    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 21:04:53.559184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.559059    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 21:04:53.570964 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.570940    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"]
Sep 10 21:04:53.637089 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.637059    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"]
Sep 10 21:04:53.639835 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.639817    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.645563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.645538    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.645710 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.645620    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k226v\" (UniqueName: \"kubernetes.io/projected/158eccbd-a701-4049-96bf-398e87a3cca1-kube-api-access-k226v\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.645710 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.645660    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.661902 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.661877    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"]
Sep 10 21:04:53.744025 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.743998    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"]
Sep 10 21:04:53.746489 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746463    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.746591 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746544    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.746591 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746579    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.746591 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746581    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79j5p\" (UniqueName: \"kubernetes.io/projected/41260490-a965-4535-a1d5-b31f6c10011c-kube-api-access-79j5p\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.746765 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746615    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k226v\" (UniqueName: \"kubernetes.io/projected/158eccbd-a701-4049-96bf-398e87a3cca1-kube-api-access-k226v\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.746765 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746640    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.746765 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.746912 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746880    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.746989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.746970    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.761506 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.761482    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k226v\" (UniqueName: \"kubernetes.io/projected/158eccbd-a701-4049-96bf-398e87a3cca1-kube-api-access-k226v\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.766140 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.766118    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"]
Sep 10 21:04:53.842557 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.842499    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"]
Sep 10 21:04:53.844966 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.844950    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:53.847188 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847167    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.847288 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847225    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.847288 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847262    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79j5p\" (UniqueName: \"kubernetes.io/projected/41260490-a965-4535-a1d5-b31f6c10011c-kube-api-access-79j5p\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.847400 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847296    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.847400 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847332    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v559g\" (UniqueName: \"kubernetes.io/projected/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-kube-api-access-v559g\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.847400 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.847609 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847591    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.847717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.847669    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.859527 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.859507    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79j5p\" (UniqueName: \"kubernetes.io/projected/41260490-a965-4535-a1d5-b31f6c10011c-kube-api-access-79j5p\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.867340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.867325    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:53.875205 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.875184    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"]
Sep 10 21:04:53.948134 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948105    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:53.948358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948336    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.948434 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948393    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:53.948495 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948446    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk5bz\" (UniqueName: \"kubernetes.io/projected/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-kube-api-access-nk5bz\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:53.948549 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948492    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.948549 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948529    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:53.948645 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948597    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v559g\" (UniqueName: \"kubernetes.io/projected/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-kube-api-access-v559g\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.948754 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948733    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.948846 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.948828    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:53.958461 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:53.958439    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v559g\" (UniqueName: \"kubernetes.io/projected/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-kube-api-access-v559g\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:54.049220 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.049189    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:54.049372 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.049243    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk5bz\" (UniqueName: \"kubernetes.io/projected/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-kube-api-access-nk5bz\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:54.049372 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.049295    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:54.049656 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.049634    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:54.049717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.049659    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:54.055097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.055076    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:54.060097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.060069    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk5bz\" (UniqueName: \"kubernetes.io/projected/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-kube-api-access-nk5bz\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:54.144937 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.144914    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"]
Sep 10 21:04:54.146756 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:04:54.146729    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158eccbd_a701_4049_96bf_398e87a3cca1.slice/crio-03a570a8295ff7ac282175377ca526e4af62370a67a87cc365a92cbf4f2341c8 WatchSource:0}: Error finding container 03a570a8295ff7ac282175377ca526e4af62370a67a87cc365a92cbf4f2341c8: Status 404 returned error can't find the container with id 03a570a8295ff7ac282175377ca526e4af62370a67a87cc365a92cbf4f2341c8
Sep 10 21:04:54.154456 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.154431    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:54.227987 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.227927    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"]
Sep 10 21:04:54.230815 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:04:54.230786    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41260490_a965_4535_a1d5_b31f6c10011c.slice/crio-bca1ebece273ec875507ae3aa03240773fdcd5ecacf1e08e57f43ed9f2e193c5 WatchSource:0}: Error finding container bca1ebece273ec875507ae3aa03240773fdcd5ecacf1e08e57f43ed9f2e193c5: Status 404 returned error can't find the container with id bca1ebece273ec875507ae3aa03240773fdcd5ecacf1e08e57f43ed9f2e193c5
Sep 10 21:04:54.340517 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.340490    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"]
Sep 10 21:04:54.345129 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.345103    2557 generic.go:356] "Generic (PLEG): container finished" podID="158eccbd-a701-4049-96bf-398e87a3cca1" containerID="bf63dccf14c9800447efd9e22a790f2d2066ca63561748ae59d66bd59f7c2f2b" exitCode=0
Sep 10 21:04:54.345230 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.345179    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7" event={"ID":"158eccbd-a701-4049-96bf-398e87a3cca1","Type":"ContainerDied","Data":"bf63dccf14c9800447efd9e22a790f2d2066ca63561748ae59d66bd59f7c2f2b"}
Sep 10 21:04:54.345230 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.345226    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7" event={"ID":"158eccbd-a701-4049-96bf-398e87a3cca1","Type":"ContainerStarted","Data":"03a570a8295ff7ac282175377ca526e4af62370a67a87cc365a92cbf4f2341c8"}
Sep 10 21:04:54.346286 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.346268    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:04:54.346982 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.346961    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x" event={"ID":"41260490-a965-4535-a1d5-b31f6c10011c","Type":"ContainerStarted","Data":"7221d8c50230d178afbd36c60b6e629c91060c93c59350c5b16ddfbd79e66fcd"}
Sep 10 21:04:54.347061 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.347011    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x" event={"ID":"41260490-a965-4535-a1d5-b31f6c10011c","Type":"ContainerStarted","Data":"bca1ebece273ec875507ae3aa03240773fdcd5ecacf1e08e57f43ed9f2e193c5"}
Sep 10 21:04:54.395497 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:04:54.395443    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a07ffd_d1e5_4be4_9b84_4c51cd51b3fd.slice/crio-3c3220a983ce0499dac0d25243a9468a630b19df9653bc5858eab74339945078 WatchSource:0}: Error finding container 3c3220a983ce0499dac0d25243a9468a630b19df9653bc5858eab74339945078: Status 404 returned error can't find the container with id 3c3220a983ce0499dac0d25243a9468a630b19df9653bc5858eab74339945078
Sep 10 21:04:54.459229 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:54.459204    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"]
Sep 10 21:04:54.460730 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:04:54.460700    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3a2f5b_5a5c_4c7b_9f42_a20eb604ba3b.slice/crio-e0dc25c6a2d9a42cc412bcfef9a016ff77ae5fc06f3ae7c6162d38c9347df3c5 WatchSource:0}: Error finding container e0dc25c6a2d9a42cc412bcfef9a016ff77ae5fc06f3ae7c6162d38c9347df3c5: Status 404 returned error can't find the container with id e0dc25c6a2d9a42cc412bcfef9a016ff77ae5fc06f3ae7c6162d38c9347df3c5
Sep 10 21:04:55.353016 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.352982    2557 generic.go:356] "Generic (PLEG): container finished" podID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerID="0bf242f4b302f9a57efcec8e8d1d8451cd383fd4f495be2be9c0b1b7327eab3b" exitCode=0
Sep 10 21:04:55.353469 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.353068    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2" event={"ID":"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b","Type":"ContainerDied","Data":"0bf242f4b302f9a57efcec8e8d1d8451cd383fd4f495be2be9c0b1b7327eab3b"}
Sep 10 21:04:55.353469 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.353132    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2" event={"ID":"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b","Type":"ContainerStarted","Data":"e0dc25c6a2d9a42cc412bcfef9a016ff77ae5fc06f3ae7c6162d38c9347df3c5"}
Sep 10 21:04:55.359588 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.359563    2557 generic.go:356] "Generic (PLEG): container finished" podID="158eccbd-a701-4049-96bf-398e87a3cca1" containerID="ea3d5b2000f77ac24ab13b745e25cc04aed303e769966e3afcdeae18dfbd5cfa" exitCode=0
Sep 10 21:04:55.359717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.359645    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7" event={"ID":"158eccbd-a701-4049-96bf-398e87a3cca1","Type":"ContainerDied","Data":"ea3d5b2000f77ac24ab13b745e25cc04aed303e769966e3afcdeae18dfbd5cfa"}
Sep 10 21:04:55.361592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.361569    2557 generic.go:356] "Generic (PLEG): container finished" podID="41260490-a965-4535-a1d5-b31f6c10011c" containerID="7221d8c50230d178afbd36c60b6e629c91060c93c59350c5b16ddfbd79e66fcd" exitCode=0
Sep 10 21:04:55.361592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.361590    2557 generic.go:356] "Generic (PLEG): container finished" podID="41260490-a965-4535-a1d5-b31f6c10011c" containerID="4098a2700672e902ce737cd4ae774097d1c6d46c775de53671e10cd0eece9df6" exitCode=0
Sep 10 21:04:55.361771 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.361643    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x" event={"ID":"41260490-a965-4535-a1d5-b31f6c10011c","Type":"ContainerDied","Data":"7221d8c50230d178afbd36c60b6e629c91060c93c59350c5b16ddfbd79e66fcd"}
Sep 10 21:04:55.361771 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.361679    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x" event={"ID":"41260490-a965-4535-a1d5-b31f6c10011c","Type":"ContainerDied","Data":"4098a2700672e902ce737cd4ae774097d1c6d46c775de53671e10cd0eece9df6"}
Sep 10 21:04:55.363107 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.363086    2557 generic.go:356] "Generic (PLEG): container finished" podID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerID="b2c6bd56120f0214e81b058b9c443af4fb2700fcb2f906d4116a3180e616d71a" exitCode=0
Sep 10 21:04:55.363200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.363159    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx" event={"ID":"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd","Type":"ContainerDied","Data":"b2c6bd56120f0214e81b058b9c443af4fb2700fcb2f906d4116a3180e616d71a"}
Sep 10 21:04:55.363270 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:55.363204    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx" event={"ID":"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd","Type":"ContainerStarted","Data":"3c3220a983ce0499dac0d25243a9468a630b19df9653bc5858eab74339945078"}
Sep 10 21:04:56.369990 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.369959    2557 generic.go:356] "Generic (PLEG): container finished" podID="41260490-a965-4535-a1d5-b31f6c10011c" containerID="e465144ceb4e8b9def5cf6f575177400981c75de839d74c3190f09df527ba9fe" exitCode=0
Sep 10 21:04:56.370419 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.370038    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x" event={"ID":"41260490-a965-4535-a1d5-b31f6c10011c","Type":"ContainerDied","Data":"e465144ceb4e8b9def5cf6f575177400981c75de839d74c3190f09df527ba9fe"}
Sep 10 21:04:56.371770 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.371749    2557 generic.go:356] "Generic (PLEG): container finished" podID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerID="aa0336c0e9bf16a83728e6f23bfac95f5a8f6f30db4c47ef20ea561320788cea" exitCode=0
Sep 10 21:04:56.371890 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.371831    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx" event={"ID":"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd","Type":"ContainerDied","Data":"aa0336c0e9bf16a83728e6f23bfac95f5a8f6f30db4c47ef20ea561320788cea"}
Sep 10 21:04:56.373804 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.373779    2557 generic.go:356] "Generic (PLEG): container finished" podID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerID="119b0500941827d3812cc9976141220e3c4e0832460901f3511c7ded71256424" exitCode=0
Sep 10 21:04:56.373880 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.373858    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2" event={"ID":"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b","Type":"ContainerDied","Data":"119b0500941827d3812cc9976141220e3c4e0832460901f3511c7ded71256424"}
Sep 10 21:04:56.376340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.376314    2557 generic.go:356] "Generic (PLEG): container finished" podID="158eccbd-a701-4049-96bf-398e87a3cca1" containerID="05a876ab6d674dcde1a7ce0c66edd089cb5591ab96b17498afb405e269727f41" exitCode=0
Sep 10 21:04:56.376401 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:56.376345    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7" event={"ID":"158eccbd-a701-4049-96bf-398e87a3cca1","Type":"ContainerDied","Data":"05a876ab6d674dcde1a7ce0c66edd089cb5591ab96b17498afb405e269727f41"}
Sep 10 21:04:57.384131 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.384092    2557 generic.go:356] "Generic (PLEG): container finished" podID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerID="3ac5c8c0da7314d9054b9c73f793803812519c3244b248a9cf2b991e982ba8be" exitCode=0
Sep 10 21:04:57.384557 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.384173    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx" event={"ID":"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd","Type":"ContainerDied","Data":"3ac5c8c0da7314d9054b9c73f793803812519c3244b248a9cf2b991e982ba8be"}
Sep 10 21:04:57.386262 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.386238    2557 generic.go:356] "Generic (PLEG): container finished" podID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerID="643fcbf8fff927bd2a484b2070c82ee62005b305566b591c18d0ef95795a9723" exitCode=0
Sep 10 21:04:57.386399 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.386312    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2" event={"ID":"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b","Type":"ContainerDied","Data":"643fcbf8fff927bd2a484b2070c82ee62005b305566b591c18d0ef95795a9723"}
Sep 10 21:04:57.516534 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.516511    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:57.547197 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.547174    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:57.580825 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.580794    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k226v\" (UniqueName: \"kubernetes.io/projected/158eccbd-a701-4049-96bf-398e87a3cca1-kube-api-access-k226v\") pod \"158eccbd-a701-4049-96bf-398e87a3cca1\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") "
Sep 10 21:04:57.580953 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.580926    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-util\") pod \"158eccbd-a701-4049-96bf-398e87a3cca1\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") "
Sep 10 21:04:57.580997 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.580952    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-bundle\") pod \"158eccbd-a701-4049-96bf-398e87a3cca1\" (UID: \"158eccbd-a701-4049-96bf-398e87a3cca1\") "
Sep 10 21:04:57.581481 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.581451    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-bundle" (OuterVolumeSpecName: "bundle") pod "158eccbd-a701-4049-96bf-398e87a3cca1" (UID: "158eccbd-a701-4049-96bf-398e87a3cca1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:57.582757 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.582728    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/158eccbd-a701-4049-96bf-398e87a3cca1-kube-api-access-k226v" (OuterVolumeSpecName: "kube-api-access-k226v") pod "158eccbd-a701-4049-96bf-398e87a3cca1" (UID: "158eccbd-a701-4049-96bf-398e87a3cca1"). InnerVolumeSpecName "kube-api-access-k226v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:04:57.586342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.586317    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-util" (OuterVolumeSpecName: "util") pod "158eccbd-a701-4049-96bf-398e87a3cca1" (UID: "158eccbd-a701-4049-96bf-398e87a3cca1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:57.682223 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.682162    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79j5p\" (UniqueName: \"kubernetes.io/projected/41260490-a965-4535-a1d5-b31f6c10011c-kube-api-access-79j5p\") pod \"41260490-a965-4535-a1d5-b31f6c10011c\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") "
Sep 10 21:04:57.682330 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.682263    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-bundle\") pod \"41260490-a965-4535-a1d5-b31f6c10011c\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") "
Sep 10 21:04:57.682330 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.682292    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-util\") pod \"41260490-a965-4535-a1d5-b31f6c10011c\" (UID: \"41260490-a965-4535-a1d5-b31f6c10011c\") "
Sep 10 21:04:57.682885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.682862    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k226v\" (UniqueName: \"kubernetes.io/projected/158eccbd-a701-4049-96bf-398e87a3cca1-kube-api-access-k226v\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:57.682976 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.682903    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:57.682976 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.682926    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/158eccbd-a701-4049-96bf-398e87a3cca1-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:57.683065 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.683024    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-bundle" (OuterVolumeSpecName: "bundle") pod "41260490-a965-4535-a1d5-b31f6c10011c" (UID: "41260490-a965-4535-a1d5-b31f6c10011c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:57.684268 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.684244    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41260490-a965-4535-a1d5-b31f6c10011c-kube-api-access-79j5p" (OuterVolumeSpecName: "kube-api-access-79j5p") pod "41260490-a965-4535-a1d5-b31f6c10011c" (UID: "41260490-a965-4535-a1d5-b31f6c10011c"). InnerVolumeSpecName "kube-api-access-79j5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:04:57.688195 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.688173    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-util" (OuterVolumeSpecName: "util") pod "41260490-a965-4535-a1d5-b31f6c10011c" (UID: "41260490-a965-4535-a1d5-b31f6c10011c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:57.783937 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.783914    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79j5p\" (UniqueName: \"kubernetes.io/projected/41260490-a965-4535-a1d5-b31f6c10011c-kube-api-access-79j5p\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:57.783937 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.783937    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:57.784043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:57.783947    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41260490-a965-4535-a1d5-b31f6c10011c-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:58.393586 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.393556    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7"
Sep 10 21:04:58.394036 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.393583    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7wslg7" event={"ID":"158eccbd-a701-4049-96bf-398e87a3cca1","Type":"ContainerDied","Data":"03a570a8295ff7ac282175377ca526e4af62370a67a87cc365a92cbf4f2341c8"}
Sep 10 21:04:58.394036 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.393633    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03a570a8295ff7ac282175377ca526e4af62370a67a87cc365a92cbf4f2341c8"
Sep 10 21:04:58.395504 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.395476    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x" event={"ID":"41260490-a965-4535-a1d5-b31f6c10011c","Type":"ContainerDied","Data":"bca1ebece273ec875507ae3aa03240773fdcd5ecacf1e08e57f43ed9f2e193c5"}
Sep 10 21:04:58.395642 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.395528    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bca1ebece273ec875507ae3aa03240773fdcd5ecacf1e08e57f43ed9f2e193c5"
Sep 10 21:04:58.395642 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.395493    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847an5c9x"
Sep 10 21:04:58.544465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.544440    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:58.546649 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.546633    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:58.691154 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691074    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk5bz\" (UniqueName: \"kubernetes.io/projected/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-kube-api-access-nk5bz\") pod \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") "
Sep 10 21:04:58.691154 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691117    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-util\") pod \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") "
Sep 10 21:04:58.691154 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691148    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-util\") pod \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") "
Sep 10 21:04:58.691407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691189    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v559g\" (UniqueName: \"kubernetes.io/projected/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-kube-api-access-v559g\") pod \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") "
Sep 10 21:04:58.691407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691219    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-bundle\") pod \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\" (UID: \"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd\") "
Sep 10 21:04:58.691407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691243    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-bundle\") pod \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\" (UID: \"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b\") "
Sep 10 21:04:58.691806 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691770    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-bundle" (OuterVolumeSpecName: "bundle") pod "ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" (UID: "ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:58.691806 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.691772    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-bundle" (OuterVolumeSpecName: "bundle") pod "e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" (UID: "e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:58.693252 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.693223    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-kube-api-access-v559g" (OuterVolumeSpecName: "kube-api-access-v559g") pod "e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" (UID: "e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd"). InnerVolumeSpecName "kube-api-access-v559g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:04:58.693353 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.693258    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-kube-api-access-nk5bz" (OuterVolumeSpecName: "kube-api-access-nk5bz") pod "ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" (UID: "ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b"). InnerVolumeSpecName "kube-api-access-nk5bz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:04:58.700351 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.700324    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-util" (OuterVolumeSpecName: "util") pod "e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" (UID: "e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:58.700428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.700409    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-util" (OuterVolumeSpecName: "util") pod "ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" (UID: "ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:04:58.792517 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.792491    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:58.792517 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.792514    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v559g\" (UniqueName: \"kubernetes.io/projected/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-kube-api-access-v559g\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:58.792656 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.792530    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:58.792656 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.792543    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-bundle\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:58.792656 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.792555    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk5bz\" (UniqueName: \"kubernetes.io/projected/ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b-kube-api-access-nk5bz\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:58.792656 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:58.792567    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd-util\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:04:59.402022 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:59.401986    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx"
Sep 10 21:04:59.402022 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:59.402000    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cdp7lx" event={"ID":"e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd","Type":"ContainerDied","Data":"3c3220a983ce0499dac0d25243a9468a630b19df9653bc5858eab74339945078"}
Sep 10 21:04:59.402453 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:59.402052    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c3220a983ce0499dac0d25243a9468a630b19df9653bc5858eab74339945078"
Sep 10 21:04:59.404064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:59.404038    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2" event={"ID":"ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b","Type":"ContainerDied","Data":"e0dc25c6a2d9a42cc412bcfef9a016ff77ae5fc06f3ae7c6162d38c9347df3c5"}
Sep 10 21:04:59.404064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:59.404049    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c676ln2"
Sep 10 21:04:59.404218 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:04:59.404071    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0dc25c6a2d9a42cc412bcfef9a016ff77ae5fc06f3ae7c6162d38c9347df3c5"
Sep 10 21:05:29.911790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.911753    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"]
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912147    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerName="pull"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912161    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerName="pull"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912170    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41260490-a965-4535-a1d5-b31f6c10011c" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912175    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41260490-a965-4535-a1d5-b31f6c10011c" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912185    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912193    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912203    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158eccbd-a701-4049-96bf-398e87a3cca1" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912209    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="158eccbd-a701-4049-96bf-398e87a3cca1" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912216    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41260490-a965-4535-a1d5-b31f6c10011c" containerName="extract"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912220    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41260490-a965-4535-a1d5-b31f6c10011c" containerName="extract"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912226    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912231    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerName="util"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912249    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41260490-a965-4535-a1d5-b31f6c10011c" containerName="pull"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912254    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41260490-a965-4535-a1d5-b31f6c10011c" containerName="pull"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912259    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerName="extract"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912264    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerName="extract"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912271    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerName="pull"
Sep 10 21:05:29.912267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912276    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerName="pull"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912288    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158eccbd-a701-4049-96bf-398e87a3cca1" containerName="extract"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912293    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="158eccbd-a701-4049-96bf-398e87a3cca1" containerName="extract"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912300    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158eccbd-a701-4049-96bf-398e87a3cca1" containerName="pull"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912305    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="158eccbd-a701-4049-96bf-398e87a3cca1" containerName="pull"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912312    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerName="extract"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912316    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerName="extract"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912372    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab3a2f5b-5a5c-4c7b-9f42-a20eb604ba3b" containerName="extract"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912379    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9a07ffd-d1e5-4be4-9b84-4c51cd51b3fd" containerName="extract"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912388    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41260490-a965-4535-a1d5-b31f6c10011c" containerName="extract"
Sep 10 21:05:29.912879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.912395    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="158eccbd-a701-4049-96bf-398e87a3cca1" containerName="extract"
Sep 10 21:05:29.915803 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.915788    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:29.924516 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.924451    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:05:29.924591 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.924571    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:05:29.924810 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.924787    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jjpzr\""
Sep 10 21:05:29.926873 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.926847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlbkd\" (UniqueName: \"kubernetes.io/projected/df0ae6ba-0ece-40f9-ab34-322d8d56757b-kube-api-access-dlbkd\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:29.926989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.926907    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df0ae6ba-0ece-40f9-ab34-322d8d56757b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:29.946588 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:29.946562    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"]
Sep 10 21:05:30.027504 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.027470    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df0ae6ba-0ece-40f9-ab34-322d8d56757b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:30.027659 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.027544    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlbkd\" (UniqueName: \"kubernetes.io/projected/df0ae6ba-0ece-40f9-ab34-322d8d56757b-kube-api-access-dlbkd\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:30.027951 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.027931    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df0ae6ba-0ece-40f9-ab34-322d8d56757b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:30.042823 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.042795    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlbkd\" (UniqueName: \"kubernetes.io/projected/df0ae6ba-0ece-40f9-ab34-322d8d56757b-kube-api-access-dlbkd\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:30.229097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.229069    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:30.439184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.439150    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"]
Sep 10 21:05:30.451586 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.451558    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"]
Sep 10 21:05:30.485883 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.485804    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"]
Sep 10 21:05:30.488987 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.488960    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:30.510304 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.510278    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"]
Sep 10 21:05:30.530495 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.530461    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219899cc-1707-452e-bb7a-c528d7e47d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-j8rfn\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:30.530648 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.530514    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkbhc\" (UniqueName: \"kubernetes.io/projected/219899cc-1707-452e-bb7a-c528d7e47d6c-kube-api-access-fkbhc\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-j8rfn\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:30.621515 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:05:30.621477    2557 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 10 21:05:30.621515 ip-10-0-140-211 kubenswrapper[2557]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-942v6_kuadrant-system_df0ae6ba-0ece-40f9-ab34-322d8d56757b_0(d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-942v6 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564" Netns:"/var/run/netns/75f74385-1633-4cb3-ab15-ec42534c3e8e" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-942v6;K8S_POD_INFRA_CONTAINER_ID=d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564;K8S_POD_UID=df0ae6ba-0ece-40f9-ab34-322d8d56757b" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6/df0ae6ba-0ece-40f9-ab34-322d8d56757b:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6 d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:24 [10.134.0.36/23]
Sep 10 21:05:30.621515 ip-10-0-140-211 kubenswrapper[2557]:         '
Sep 10 21:05:30.621515 ip-10-0-140-211 kubenswrapper[2557]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 21:05:30.621515 ip-10-0-140-211 kubenswrapper[2557]:  >
Sep 10 21:05:30.621790 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:05:30.621539    2557 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 10 21:05:30.621790 ip-10-0-140-211 kubenswrapper[2557]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-942v6_kuadrant-system_df0ae6ba-0ece-40f9-ab34-322d8d56757b_0(d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-942v6 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564" Netns:"/var/run/netns/75f74385-1633-4cb3-ab15-ec42534c3e8e" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-942v6;K8S_POD_INFRA_CONTAINER_ID=d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564;K8S_POD_UID=df0ae6ba-0ece-40f9-ab34-322d8d56757b" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6/df0ae6ba-0ece-40f9-ab34-322d8d56757b:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6 d772ceeac6d4e3fe13ef8a8760a6be7f3e8e7261396f9e37647a0bd8081ff564 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:24 [10.134.0.36/23]
Sep 10 21:05:30.621790 ip-10-0-140-211 kubenswrapper[2557]:         '
Sep 10 21:05:30.621790 ip-10-0-140-211 kubenswrapper[2557]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 21:05:30.621790 ip-10-0-140-211 kubenswrapper[2557]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:30.631580 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.631551    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219899cc-1707-452e-bb7a-c528d7e47d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-j8rfn\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:30.631702 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.631614    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkbhc\" (UniqueName: \"kubernetes.io/projected/219899cc-1707-452e-bb7a-c528d7e47d6c-kube-api-access-fkbhc\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-j8rfn\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:30.631914 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.631894    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219899cc-1707-452e-bb7a-c528d7e47d6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-j8rfn\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:30.654575 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.654545    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkbhc\" (UniqueName: \"kubernetes.io/projected/219899cc-1707-452e-bb7a-c528d7e47d6c-kube-api-access-fkbhc\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-j8rfn\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:30.803602 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:30.803529    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:31.090431 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.090401    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"]
Sep 10 21:05:31.090778 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:05:31.090583    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod219899cc_1707_452e_bb7a_c528d7e47d6c.slice/crio-f66f2798e48b8d00cd9ce964dd67fbd94ddf9f52b6d58f853365978125ea34d8 WatchSource:0}: Error finding container f66f2798e48b8d00cd9ce964dd67fbd94ddf9f52b6d58f853365978125ea34d8: Status 404 returned error can't find the container with id f66f2798e48b8d00cd9ce964dd67fbd94ddf9f52b6d58f853365978125ea34d8
Sep 10 21:05:31.567211 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.567164    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn" event={"ID":"219899cc-1707-452e-bb7a-c528d7e47d6c","Type":"ContainerStarted","Data":"f66f2798e48b8d00cd9ce964dd67fbd94ddf9f52b6d58f853365978125ea34d8"}
Sep 10 21:05:31.567211 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.567196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:31.573794 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.573772    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:31.575995 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:05:31.575970    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="df0ae6ba-0ece-40f9-ab34-322d8d56757b" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:31.739602 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.739568    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlbkd\" (UniqueName: \"kubernetes.io/projected/df0ae6ba-0ece-40f9-ab34-322d8d56757b-kube-api-access-dlbkd\") pod \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") "
Sep 10 21:05:31.739785 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.739637    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df0ae6ba-0ece-40f9-ab34-322d8d56757b-extensions-socket-volume\") pod \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\" (UID: \"df0ae6ba-0ece-40f9-ab34-322d8d56757b\") "
Sep 10 21:05:31.739929 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.739897    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df0ae6ba-0ece-40f9-ab34-322d8d56757b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "df0ae6ba-0ece-40f9-ab34-322d8d56757b" (UID: "df0ae6ba-0ece-40f9-ab34-322d8d56757b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:05:31.742415 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.742387    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0ae6ba-0ece-40f9-ab34-322d8d56757b-kube-api-access-dlbkd" (OuterVolumeSpecName: "kube-api-access-dlbkd") pod "df0ae6ba-0ece-40f9-ab34-322d8d56757b" (UID: "df0ae6ba-0ece-40f9-ab34-322d8d56757b"). InnerVolumeSpecName "kube-api-access-dlbkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:05:31.841110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.841037    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df0ae6ba-0ece-40f9-ab34-322d8d56757b-extensions-socket-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:05:31.841110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:31.841072    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlbkd\" (UniqueName: \"kubernetes.io/projected/df0ae6ba-0ece-40f9-ab34-322d8d56757b-kube-api-access-dlbkd\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:05:32.578970 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:32.578940    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:32.582337 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:05:32.582297    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-942v6\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="df0ae6ba-0ece-40f9-ab34-322d8d56757b" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-942v6"
Sep 10 21:05:32.588108 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:32.588078    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df0ae6ba-0ece-40f9-ab34-322d8d56757b" path="/var/lib/kubelet/pods/df0ae6ba-0ece-40f9-ab34-322d8d56757b/volumes"
Sep 10 21:05:35.602563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:35.602528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn" event={"ID":"219899cc-1707-452e-bb7a-c528d7e47d6c","Type":"ContainerStarted","Data":"f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b"}
Sep 10 21:05:35.602953 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:35.602649    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:35.627485 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:35.627436    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn" podStartSLOduration=1.980114104 podStartE2EDuration="5.627421681s" totalImagesPullingTime="3.647307577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:05:35.624507324 +0000 UTC m=+415.643912641" watchObservedRunningTime="2026-09-10 21:05:35.627421681 +0000 UTC m=+415.646827076"
Sep 10 21:05:46.609982 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:46.609947    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:59.154093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.154057    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"]
Sep 10 21:05:59.154457 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.154279    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn" podUID="219899cc-1707-452e-bb7a-c528d7e47d6c" containerName="manager" containerID="cri-o://f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b" gracePeriod=10
Sep 10 21:05:59.405770 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.405717    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:59.460618 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.460592    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219899cc-1707-452e-bb7a-c528d7e47d6c-extensions-socket-volume\") pod \"219899cc-1707-452e-bb7a-c528d7e47d6c\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") "
Sep 10 21:05:59.460787 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.460710    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkbhc\" (UniqueName: \"kubernetes.io/projected/219899cc-1707-452e-bb7a-c528d7e47d6c-kube-api-access-fkbhc\") pod \"219899cc-1707-452e-bb7a-c528d7e47d6c\" (UID: \"219899cc-1707-452e-bb7a-c528d7e47d6c\") "
Sep 10 21:05:59.460987 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.460966    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/219899cc-1707-452e-bb7a-c528d7e47d6c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "219899cc-1707-452e-bb7a-c528d7e47d6c" (UID: "219899cc-1707-452e-bb7a-c528d7e47d6c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:05:59.462716 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.462694    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/219899cc-1707-452e-bb7a-c528d7e47d6c-kube-api-access-fkbhc" (OuterVolumeSpecName: "kube-api-access-fkbhc") pod "219899cc-1707-452e-bb7a-c528d7e47d6c" (UID: "219899cc-1707-452e-bb7a-c528d7e47d6c"). InnerVolumeSpecName "kube-api-access-fkbhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:05:59.561724 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.561702    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/219899cc-1707-452e-bb7a-c528d7e47d6c-extensions-socket-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:05:59.561724 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.561723    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkbhc\" (UniqueName: \"kubernetes.io/projected/219899cc-1707-452e-bb7a-c528d7e47d6c-kube-api-access-fkbhc\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:05:59.725042 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.725012    2557 generic.go:356] "Generic (PLEG): container finished" podID="219899cc-1707-452e-bb7a-c528d7e47d6c" containerID="f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b" exitCode=0
Sep 10 21:05:59.725179 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.725060    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"
Sep 10 21:05:59.725179 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.725070    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn" event={"ID":"219899cc-1707-452e-bb7a-c528d7e47d6c","Type":"ContainerDied","Data":"f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b"}
Sep 10 21:05:59.725179 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.725122    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn" event={"ID":"219899cc-1707-452e-bb7a-c528d7e47d6c","Type":"ContainerDied","Data":"f66f2798e48b8d00cd9ce964dd67fbd94ddf9f52b6d58f853365978125ea34d8"}
Sep 10 21:05:59.725179 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.725153    2557 scope.go:117] "RemoveContainer" containerID="f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b"
Sep 10 21:05:59.734237 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.734222    2557 scope.go:117] "RemoveContainer" containerID="f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b"
Sep 10 21:05:59.734497 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:05:59.734479    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b\": container with ID starting with f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b not found: ID does not exist" containerID="f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b"
Sep 10 21:05:59.734538 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.734503    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b"} err="failed to get container status \"f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b\": rpc error: code = NotFound desc = could not find container \"f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b\": container with ID starting with f836e9b40cdb654b56c22d8976496abad74d92db216b2dc898320d15a8d87d0b not found: ID does not exist"
Sep 10 21:05:59.748399 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.748375    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"]
Sep 10 21:05:59.752711 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:05:59.752678    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-j8rfn"]
Sep 10 21:06:00.585097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:00.585056    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="219899cc-1707-452e-bb7a-c528d7e47d6c" path="/var/lib/kubelet/pods/219899cc-1707-452e-bb7a-c528d7e47d6c/volumes"
Sep 10 21:06:17.226074 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.225978    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:06:17.226594 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.226443    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="219899cc-1707-452e-bb7a-c528d7e47d6c" containerName="manager"
Sep 10 21:06:17.226594 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.226460    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="219899cc-1707-452e-bb7a-c528d7e47d6c" containerName="manager"
Sep 10 21:06:17.226594 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.226561    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="219899cc-1707-452e-bb7a-c528d7e47d6c" containerName="manager"
Sep 10 21:06:17.231137 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.231113    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.234534 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.234510    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:06:17.234634 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.234562    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:06:17.235359 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.235341    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pmnkr\""
Sep 10 21:06:17.235457 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.235347    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 21:06:17.241598 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.241578    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:06:17.273885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.273863    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:06:17.300263 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.300239    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbkbj\" (UniqueName: \"kubernetes.io/projected/4e95b1d0-713a-4951-8fa3-42800a2629f5-kube-api-access-qbkbj\") pod \"limitador-limitador-84994f7ffc-xqb2j\" (UID: \"4e95b1d0-713a-4951-8fa3-42800a2629f5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.300350 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.300274    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e95b1d0-713a-4951-8fa3-42800a2629f5-config-file\") pod \"limitador-limitador-84994f7ffc-xqb2j\" (UID: \"4e95b1d0-713a-4951-8fa3-42800a2629f5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.401446 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.401419    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbkbj\" (UniqueName: \"kubernetes.io/projected/4e95b1d0-713a-4951-8fa3-42800a2629f5-kube-api-access-qbkbj\") pod \"limitador-limitador-84994f7ffc-xqb2j\" (UID: \"4e95b1d0-713a-4951-8fa3-42800a2629f5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.401570 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.401454    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e95b1d0-713a-4951-8fa3-42800a2629f5-config-file\") pod \"limitador-limitador-84994f7ffc-xqb2j\" (UID: \"4e95b1d0-713a-4951-8fa3-42800a2629f5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.402054 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.402037    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e95b1d0-713a-4951-8fa3-42800a2629f5-config-file\") pod \"limitador-limitador-84994f7ffc-xqb2j\" (UID: \"4e95b1d0-713a-4951-8fa3-42800a2629f5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.411253 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.411225    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbkbj\" (UniqueName: \"kubernetes.io/projected/4e95b1d0-713a-4951-8fa3-42800a2629f5-kube-api-access-qbkbj\") pod \"limitador-limitador-84994f7ffc-xqb2j\" (UID: \"4e95b1d0-713a-4951-8fa3-42800a2629f5\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.541973 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.541915    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:17.584243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.584215    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x9mtd"]
Sep 10 21:06:17.586407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.586390    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd"
Sep 10 21:06:17.589960 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.589938    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6hnqb\""
Sep 10 21:06:17.601518 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.601475    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x9mtd"]
Sep 10 21:06:17.704587 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.704541    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75nws\" (UniqueName: \"kubernetes.io/projected/0148cf90-583b-47dc-acc6-de991b55659a-kube-api-access-75nws\") pod \"authorino-7bcfd8f9cd-x9mtd\" (UID: \"0148cf90-583b-47dc-acc6-de991b55659a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd"
Sep 10 21:06:17.784810 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.784782    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-v8rrm"]
Sep 10 21:06:17.787346 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.787327    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v8rrm"
Sep 10 21:06:17.800081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.800059    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v8rrm"]
Sep 10 21:06:17.805972 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.805947    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75nws\" (UniqueName: \"kubernetes.io/projected/0148cf90-583b-47dc-acc6-de991b55659a-kube-api-access-75nws\") pod \"authorino-7bcfd8f9cd-x9mtd\" (UID: \"0148cf90-583b-47dc-acc6-de991b55659a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd"
Sep 10 21:06:17.814723 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.814697    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75nws\" (UniqueName: \"kubernetes.io/projected/0148cf90-583b-47dc-acc6-de991b55659a-kube-api-access-75nws\") pod \"authorino-7bcfd8f9cd-x9mtd\" (UID: \"0148cf90-583b-47dc-acc6-de991b55659a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd"
Sep 10 21:06:17.824965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.824942    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:06:17.826469 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:06:17.826449    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e95b1d0_713a_4951_8fa3_42800a2629f5.slice/crio-3adf76705f817d7490f8a9dacdaf08382ac60aaabf023c3a073e9448f7c9a56d WatchSource:0}: Error finding container 3adf76705f817d7490f8a9dacdaf08382ac60aaabf023c3a073e9448f7c9a56d: Status 404 returned error can't find the container with id 3adf76705f817d7490f8a9dacdaf08382ac60aaabf023c3a073e9448f7c9a56d
Sep 10 21:06:17.907429 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.907403    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cddbm\" (UniqueName: \"kubernetes.io/projected/b586dedb-e8dc-451e-864c-9612cc0c3519-kube-api-access-cddbm\") pod \"authorino-59c974bd48-v8rrm\" (UID: \"b586dedb-e8dc-451e-864c-9612cc0c3519\") " pod="kuadrant-system/authorino-59c974bd48-v8rrm"
Sep 10 21:06:17.913307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:17.913285    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd"
Sep 10 21:06:18.008216 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.008183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cddbm\" (UniqueName: \"kubernetes.io/projected/b586dedb-e8dc-451e-864c-9612cc0c3519-kube-api-access-cddbm\") pod \"authorino-59c974bd48-v8rrm\" (UID: \"b586dedb-e8dc-451e-864c-9612cc0c3519\") " pod="kuadrant-system/authorino-59c974bd48-v8rrm"
Sep 10 21:06:18.016881 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.016859    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cddbm\" (UniqueName: \"kubernetes.io/projected/b586dedb-e8dc-451e-864c-9612cc0c3519-kube-api-access-cddbm\") pod \"authorino-59c974bd48-v8rrm\" (UID: \"b586dedb-e8dc-451e-864c-9612cc0c3519\") " pod="kuadrant-system/authorino-59c974bd48-v8rrm"
Sep 10 21:06:18.095933 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.095884    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v8rrm"
Sep 10 21:06:18.180277 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.180254    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x9mtd"]
Sep 10 21:06:18.181965 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:06:18.181941    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0148cf90_583b_47dc_acc6_de991b55659a.slice/crio-11f6191dfe694d3fcb7bb7dbf058b60370bf0a0d4d59543a034e67f71850c876 WatchSource:0}: Error finding container 11f6191dfe694d3fcb7bb7dbf058b60370bf0a0d4d59543a034e67f71850c876: Status 404 returned error can't find the container with id 11f6191dfe694d3fcb7bb7dbf058b60370bf0a0d4d59543a034e67f71850c876
Sep 10 21:06:18.363590 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.363564    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v8rrm"]
Sep 10 21:06:18.365365 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:06:18.365337    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb586dedb_e8dc_451e_864c_9612cc0c3519.slice/crio-2efdeb6fe43aec4f4f955604c61ce48277d53210849e9b2900dc59de658d8f04 WatchSource:0}: Error finding container 2efdeb6fe43aec4f4f955604c61ce48277d53210849e9b2900dc59de658d8f04: Status 404 returned error can't find the container with id 2efdeb6fe43aec4f4f955604c61ce48277d53210849e9b2900dc59de658d8f04
Sep 10 21:06:18.831260 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.831218    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j" event={"ID":"4e95b1d0-713a-4951-8fa3-42800a2629f5","Type":"ContainerStarted","Data":"3adf76705f817d7490f8a9dacdaf08382ac60aaabf023c3a073e9448f7c9a56d"}
Sep 10 21:06:18.832748 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.832720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v8rrm" event={"ID":"b586dedb-e8dc-451e-864c-9612cc0c3519","Type":"ContainerStarted","Data":"2efdeb6fe43aec4f4f955604c61ce48277d53210849e9b2900dc59de658d8f04"}
Sep 10 21:06:18.834047 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:18.834020    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd" event={"ID":"0148cf90-583b-47dc-acc6-de991b55659a","Type":"ContainerStarted","Data":"11f6191dfe694d3fcb7bb7dbf058b60370bf0a0d4d59543a034e67f71850c876"}
Sep 10 21:06:19.842227 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:19.842193    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j" event={"ID":"4e95b1d0-713a-4951-8fa3-42800a2629f5","Type":"ContainerStarted","Data":"520887383a3ed7e38ab467f973629d87e54c1745e4ca2e7b8cb27b5151ff6a82"}
Sep 10 21:06:19.842639 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:19.842351    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:19.863184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:19.863128    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j" podStartSLOduration=1.809094091 podStartE2EDuration="2.863112273s" totalImagesPullingTime="1.054018182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:19.859576786 +0000 UTC m=+459.878982129" watchObservedRunningTime="2026-09-10 21:06:19.863112273 +0000 UTC m=+459.882517591"
Sep 10 21:06:20.848801 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:20.848762    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v8rrm" event={"ID":"b586dedb-e8dc-451e-864c-9612cc0c3519","Type":"ContainerStarted","Data":"99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f"}
Sep 10 21:06:20.850384 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:20.850362    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd" event={"ID":"0148cf90-583b-47dc-acc6-de991b55659a","Type":"ContainerStarted","Data":"b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99"}
Sep 10 21:06:20.865063 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:20.865018    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-v8rrm" podStartSLOduration=2.152282152 podStartE2EDuration="3.865004851s" totalImagesPullingTime="1.712722699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:20.86339927 +0000 UTC m=+460.882804589" watchObservedRunningTime="2026-09-10 21:06:20.865004851 +0000 UTC m=+460.884410169"
Sep 10 21:06:20.878512 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:20.878463    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd" podStartSLOduration=1.986637223 podStartE2EDuration="3.87844442s" totalImagesPullingTime="1.891807197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:06:20.877089625 +0000 UTC m=+460.896494943" watchObservedRunningTime="2026-09-10 21:06:20.87844442 +0000 UTC m=+460.897849738"
Sep 10 21:06:20.905946 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:20.905916    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x9mtd"]
Sep 10 21:06:22.861761 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:22.861721    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd" podUID="0148cf90-583b-47dc-acc6-de991b55659a" containerName="authorino" containerID="cri-o://b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99" gracePeriod=30
Sep 10 21:06:23.097660 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.097633    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd"
Sep 10 21:06:23.256005 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.255977    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75nws\" (UniqueName: \"kubernetes.io/projected/0148cf90-583b-47dc-acc6-de991b55659a-kube-api-access-75nws\") pod \"0148cf90-583b-47dc-acc6-de991b55659a\" (UID: \"0148cf90-583b-47dc-acc6-de991b55659a\") "
Sep 10 21:06:23.258051 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.258020    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0148cf90-583b-47dc-acc6-de991b55659a-kube-api-access-75nws" (OuterVolumeSpecName: "kube-api-access-75nws") pod "0148cf90-583b-47dc-acc6-de991b55659a" (UID: "0148cf90-583b-47dc-acc6-de991b55659a"). InnerVolumeSpecName "kube-api-access-75nws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:06:23.356569 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.356537    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75nws\" (UniqueName: \"kubernetes.io/projected/0148cf90-583b-47dc-acc6-de991b55659a-kube-api-access-75nws\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:06:23.868068 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.868035    2557 generic.go:356] "Generic (PLEG): container finished" podID="0148cf90-583b-47dc-acc6-de991b55659a" containerID="b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99" exitCode=0
Sep 10 21:06:23.868497 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.868075    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd"
Sep 10 21:06:23.868497 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.868120    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd" event={"ID":"0148cf90-583b-47dc-acc6-de991b55659a","Type":"ContainerDied","Data":"b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99"}
Sep 10 21:06:23.868497 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.868160    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-x9mtd" event={"ID":"0148cf90-583b-47dc-acc6-de991b55659a","Type":"ContainerDied","Data":"11f6191dfe694d3fcb7bb7dbf058b60370bf0a0d4d59543a034e67f71850c876"}
Sep 10 21:06:23.868497 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.868177    2557 scope.go:117] "RemoveContainer" containerID="b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99"
Sep 10 21:06:23.877098 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.877079    2557 scope.go:117] "RemoveContainer" containerID="b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99"
Sep 10 21:06:23.877356 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:06:23.877338    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99\": container with ID starting with b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99 not found: ID does not exist" containerID="b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99"
Sep 10 21:06:23.877428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.877364    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99"} err="failed to get container status \"b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99\": rpc error: code = NotFound desc = could not find container \"b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99\": container with ID starting with b2f1e26b480d41b88835ee92b8ab19ce0f6137795b692c976859b83efac08e99 not found: ID does not exist"
Sep 10 21:06:23.892419 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.892397    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x9mtd"]
Sep 10 21:06:23.896376 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:23.896357    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-x9mtd"]
Sep 10 21:06:24.585037 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:24.585006    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0148cf90-583b-47dc-acc6-de991b55659a" path="/var/lib/kubelet/pods/0148cf90-583b-47dc-acc6-de991b55659a/volumes"
Sep 10 21:06:30.851665 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:30.851631    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"
Sep 10 21:06:47.133515 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.133482    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd777bbc8-w67hp"]
Sep 10 21:06:47.133986 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.133870    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0148cf90-583b-47dc-acc6-de991b55659a" containerName="authorino"
Sep 10 21:06:47.133986 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.133885    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0148cf90-583b-47dc-acc6-de991b55659a" containerName="authorino"
Sep 10 21:06:47.133986 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.133949    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0148cf90-583b-47dc-acc6-de991b55659a" containerName="authorino"
Sep 10 21:06:47.136437 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.136415    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd777bbc8-w67hp"
Sep 10 21:06:47.150308 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.150280    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd777bbc8-w67hp"]
Sep 10 21:06:47.240886 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.240856    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swftx\" (UniqueName: \"kubernetes.io/projected/874cda59-3e3e-4ad2-a4bd-a0ea35fd047b-kube-api-access-swftx\") pod \"authorino-6dd777bbc8-w67hp\" (UID: \"874cda59-3e3e-4ad2-a4bd-a0ea35fd047b\") " pod="kuadrant-system/authorino-6dd777bbc8-w67hp"
Sep 10 21:06:47.253942 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.253912    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd777bbc8-w67hp"]
Sep 10 21:06:47.254143 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:06:47.254119    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-swftx], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6dd777bbc8-w67hp" podUID="874cda59-3e3e-4ad2-a4bd-a0ea35fd047b"
Sep 10 21:06:47.341955 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.341923    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swftx\" (UniqueName: \"kubernetes.io/projected/874cda59-3e3e-4ad2-a4bd-a0ea35fd047b-kube-api-access-swftx\") pod \"authorino-6dd777bbc8-w67hp\" (UID: \"874cda59-3e3e-4ad2-a4bd-a0ea35fd047b\") " pod="kuadrant-system/authorino-6dd777bbc8-w67hp"
Sep 10 21:06:47.350793 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.350764    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swftx\" (UniqueName: \"kubernetes.io/projected/874cda59-3e3e-4ad2-a4bd-a0ea35fd047b-kube-api-access-swftx\") pod \"authorino-6dd777bbc8-w67hp\" (UID: \"874cda59-3e3e-4ad2-a4bd-a0ea35fd047b\") " pod="kuadrant-system/authorino-6dd777bbc8-w67hp"
Sep 10 21:06:47.994783 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:47.994753    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd777bbc8-w67hp"
Sep 10 21:06:48.000821 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:48.000797    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd777bbc8-w67hp"
Sep 10 21:06:48.046673 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:48.046645    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swftx\" (UniqueName: \"kubernetes.io/projected/874cda59-3e3e-4ad2-a4bd-a0ea35fd047b-kube-api-access-swftx\") pod \"874cda59-3e3e-4ad2-a4bd-a0ea35fd047b\" (UID: \"874cda59-3e3e-4ad2-a4bd-a0ea35fd047b\") "
Sep 10 21:06:48.048716 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:48.048664    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/874cda59-3e3e-4ad2-a4bd-a0ea35fd047b-kube-api-access-swftx" (OuterVolumeSpecName: "kube-api-access-swftx") pod "874cda59-3e3e-4ad2-a4bd-a0ea35fd047b" (UID: "874cda59-3e3e-4ad2-a4bd-a0ea35fd047b"). InnerVolumeSpecName "kube-api-access-swftx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:06:48.148205 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:48.148172    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swftx\" (UniqueName: \"kubernetes.io/projected/874cda59-3e3e-4ad2-a4bd-a0ea35fd047b-kube-api-access-swftx\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:06:49.000021 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:48.999988    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd777bbc8-w67hp"
Sep 10 21:06:49.056531 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.056502    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd777bbc8-w67hp"]
Sep 10 21:06:49.061519 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.061496    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd777bbc8-w67hp"]
Sep 10 21:06:49.469634 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.469595    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v8rrm"]
Sep 10 21:06:49.470083 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.469832    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-v8rrm" podUID="b586dedb-e8dc-451e-864c-9612cc0c3519" containerName="authorino" containerID="cri-o://99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f" gracePeriod=30
Sep 10 21:06:49.719365 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.719333    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v8rrm"
Sep 10 21:06:49.761300 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.761234    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cddbm\" (UniqueName: \"kubernetes.io/projected/b586dedb-e8dc-451e-864c-9612cc0c3519-kube-api-access-cddbm\") pod \"b586dedb-e8dc-451e-864c-9612cc0c3519\" (UID: \"b586dedb-e8dc-451e-864c-9612cc0c3519\") "
Sep 10 21:06:49.763271 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.763242    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b586dedb-e8dc-451e-864c-9612cc0c3519-kube-api-access-cddbm" (OuterVolumeSpecName: "kube-api-access-cddbm") pod "b586dedb-e8dc-451e-864c-9612cc0c3519" (UID: "b586dedb-e8dc-451e-864c-9612cc0c3519"). InnerVolumeSpecName "kube-api-access-cddbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:06:49.862149 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:49.862123    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cddbm\" (UniqueName: \"kubernetes.io/projected/b586dedb-e8dc-451e-864c-9612cc0c3519-kube-api-access-cddbm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:06:50.006451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.006421    2557 generic.go:356] "Generic (PLEG): container finished" podID="b586dedb-e8dc-451e-864c-9612cc0c3519" containerID="99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f" exitCode=0
Sep 10 21:06:50.006614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.006458    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-v8rrm"
Sep 10 21:06:50.006614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.006508    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v8rrm" event={"ID":"b586dedb-e8dc-451e-864c-9612cc0c3519","Type":"ContainerDied","Data":"99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f"}
Sep 10 21:06:50.006614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.006544    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-v8rrm" event={"ID":"b586dedb-e8dc-451e-864c-9612cc0c3519","Type":"ContainerDied","Data":"2efdeb6fe43aec4f4f955604c61ce48277d53210849e9b2900dc59de658d8f04"}
Sep 10 21:06:50.006614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.006561    2557 scope.go:117] "RemoveContainer" containerID="99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f"
Sep 10 21:06:50.015354 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.015334    2557 scope.go:117] "RemoveContainer" containerID="99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f"
Sep 10 21:06:50.015618 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:06:50.015600    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f\": container with ID starting with 99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f not found: ID does not exist" containerID="99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f"
Sep 10 21:06:50.015669 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.015628    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f"} err="failed to get container status \"99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f\": rpc error: code = NotFound desc = could not find container \"99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f\": container with ID starting with 99cd19eb197ebaa7e6e64b4296f5b3110473f21bd1e132a68be1003a4146db2f not found: ID does not exist"
Sep 10 21:06:50.039470 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.039447    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v8rrm"]
Sep 10 21:06:50.053784 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.053764    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-v8rrm"]
Sep 10 21:06:50.584858 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.584830    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="874cda59-3e3e-4ad2-a4bd-a0ea35fd047b" path="/var/lib/kubelet/pods/874cda59-3e3e-4ad2-a4bd-a0ea35fd047b/volumes"
Sep 10 21:06:50.585242 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:06:50.585054    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b586dedb-e8dc-451e-864c-9612cc0c3519" path="/var/lib/kubelet/pods/b586dedb-e8dc-451e-864c-9612cc0c3519/volumes"
Sep 10 21:07:32.687911 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.687876    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"]
Sep 10 21:07:32.688459 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.688355    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b586dedb-e8dc-451e-864c-9612cc0c3519" containerName="authorino"
Sep 10 21:07:32.688459 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.688373    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b586dedb-e8dc-451e-864c-9612cc0c3519" containerName="authorino"
Sep 10 21:07:32.688581 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.688482    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b586dedb-e8dc-451e-864c-9612cc0c3519" containerName="authorino"
Sep 10 21:07:32.691585 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.691564    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:32.694206 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.694186    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 21:07:32.694305 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.694260    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ws5rz\""
Sep 10 21:07:32.715263 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.715239    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"]
Sep 10 21:07:32.807562 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.807499    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d437794-4d5d-49d5-8005-31a16c2ee4c3-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-p7hvp\" (UID: \"6d437794-4d5d-49d5-8005-31a16c2ee4c3\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:32.807562 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.807539    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql8rr\" (UniqueName: \"kubernetes.io/projected/6d437794-4d5d-49d5-8005-31a16c2ee4c3-kube-api-access-ql8rr\") pod \"payload-processing-5b54b7fbb7-p7hvp\" (UID: \"6d437794-4d5d-49d5-8005-31a16c2ee4c3\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:32.908981 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.908952    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d437794-4d5d-49d5-8005-31a16c2ee4c3-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-p7hvp\" (UID: \"6d437794-4d5d-49d5-8005-31a16c2ee4c3\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:32.909112 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.908994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql8rr\" (UniqueName: \"kubernetes.io/projected/6d437794-4d5d-49d5-8005-31a16c2ee4c3-kube-api-access-ql8rr\") pod \"payload-processing-5b54b7fbb7-p7hvp\" (UID: \"6d437794-4d5d-49d5-8005-31a16c2ee4c3\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:32.909734 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.909714    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d437794-4d5d-49d5-8005-31a16c2ee4c3-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-p7hvp\" (UID: \"6d437794-4d5d-49d5-8005-31a16c2ee4c3\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:32.921893 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:32.921865    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql8rr\" (UniqueName: \"kubernetes.io/projected/6d437794-4d5d-49d5-8005-31a16c2ee4c3-kube-api-access-ql8rr\") pod \"payload-processing-5b54b7fbb7-p7hvp\" (UID: \"6d437794-4d5d-49d5-8005-31a16c2ee4c3\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:33.002088 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:33.002054    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:33.019534 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:33.019505    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:07:33.283807 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:33.283778    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"]
Sep 10 21:07:33.285800 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:07:33.285771    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d437794_4d5d_49d5_8005_31a16c2ee4c3.slice/crio-34cfaff7e960ef1610c700c0e33f050e08445627d699d01dd1e26bd2559cea91 WatchSource:0}: Error finding container 34cfaff7e960ef1610c700c0e33f050e08445627d699d01dd1e26bd2559cea91: Status 404 returned error can't find the container with id 34cfaff7e960ef1610c700c0e33f050e08445627d699d01dd1e26bd2559cea91
Sep 10 21:07:34.232549 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:34.232502    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp" event={"ID":"6d437794-4d5d-49d5-8005-31a16c2ee4c3","Type":"ContainerStarted","Data":"34cfaff7e960ef1610c700c0e33f050e08445627d699d01dd1e26bd2559cea91"}
Sep 10 21:07:35.239435 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:35.239393    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp" event={"ID":"6d437794-4d5d-49d5-8005-31a16c2ee4c3","Type":"ContainerStarted","Data":"b319c7086613c30356fa1f8c3f76984e01792f0a2806b478590e91d49a895aff"}
Sep 10 21:07:35.239839 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:35.239489    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:35.261432 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:35.261384    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp" podStartSLOduration=1.702331364 podStartE2EDuration="3.261370931s" totalImagesPullingTime="1.559039567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:07:35.260676414 +0000 UTC m=+535.280081734" watchObservedRunningTime="2026-09-10 21:07:35.261370931 +0000 UTC m=+535.280776280"
Sep 10 21:07:46.246096 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:46.246062    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-p7hvp"
Sep 10 21:07:50.341464 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.341427    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"]
Sep 10 21:07:50.345976 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.345957    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.348614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.348593    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 21:07:50.348746 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.348735    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 21:07:50.349661 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.349639    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 10 21:07:50.349778 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.349648    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-njkdt\""
Sep 10 21:07:50.355293 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.355269    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"]
Sep 10 21:07:50.442289 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.442264    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.442410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.442311    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.442410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.442339    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54nhm\" (UniqueName: \"kubernetes.io/projected/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-kube-api-access-54nhm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.442410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.442366    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.442410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.442380    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.442410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.442395    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543316 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543282    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543350    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543385    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54nhm\" (UniqueName: \"kubernetes.io/projected/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-kube-api-access-54nhm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543422    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543444    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543501 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543467    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543802 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543769    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543802 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543788    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.543908 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.543822    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.545752 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.545733    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.545936 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.545919    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.551614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.551591    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54nhm\" (UniqueName: \"kubernetes.io/projected/2f5b07ce-c298-46b4-af80-2a1c8ad29f36-kube-api-access-54nhm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq\" (UID: \"2f5b07ce-c298-46b4-af80-2a1c8ad29f36\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.656630 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.656568    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:07:50.930074 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.930054    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"]
Sep 10 21:07:50.933988 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:07:50.933060    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5b07ce_c298_46b4_af80_2a1c8ad29f36.slice/crio-131ee612d2612bc63e5402929a2baee6b18f49b839da638bc985b2d0821e2d90 WatchSource:0}: Error finding container 131ee612d2612bc63e5402929a2baee6b18f49b839da638bc985b2d0821e2d90: Status 404 returned error can't find the container with id 131ee612d2612bc63e5402929a2baee6b18f49b839da638bc985b2d0821e2d90
Sep 10 21:07:50.935713 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:50.935209    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:07:51.325236 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:51.325200    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq" event={"ID":"2f5b07ce-c298-46b4-af80-2a1c8ad29f36","Type":"ContainerStarted","Data":"131ee612d2612bc63e5402929a2baee6b18f49b839da638bc985b2d0821e2d90"}
Sep 10 21:07:52.332064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:52.332020    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq" event={"ID":"2f5b07ce-c298-46b4-af80-2a1c8ad29f36","Type":"ContainerStarted","Data":"79cc3da4b0a65f129fbf1723efc4464a93cedf9bc4917e8ce3dc0cdf242ac181"}
Sep 10 21:07:52.354833 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:07:52.354786    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq" podStartSLOduration=1.332835682 podStartE2EDuration="2.354769588s" totalImagesPullingTime="1.021933906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:07:52.353077632 +0000 UTC m=+552.372482953" watchObservedRunningTime="2026-09-10 21:07:52.354769588 +0000 UTC m=+552.374174905"
Sep 10 21:08:00.466298 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:00.466258    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:08:00.657629 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:00.657597    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:08:00.657817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:00.657790    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:08:00.670455 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:00.670431    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:08:01.388010 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:01.387977    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq"
Sep 10 21:08:10.694808 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:10.694774    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:08:14.336835 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:14.336799    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:08:22.542340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:22.542303    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:08:40.395306 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:40.395274    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:08:40.395717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:40.395640    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:08:44.542150 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:44.542116    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:08:49.030220 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.030182    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"]
Sep 10 21:08:49.033945 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.033922    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.036477 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.036446    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.036607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.036491    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04738a95-2480-47cd-adfd-ce3e954f9156-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.036607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.036531    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.036745 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.036663    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.036745 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.036720    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.036854 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.036744    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 21:08:49.036854 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.036769    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2w2z\" (UniqueName: \"kubernetes.io/projected/04738a95-2480-47cd-adfd-ce3e954f9156-kube-api-access-l2w2z\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.045153 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.045129    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"]
Sep 10 21:08:49.137741 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.137703    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.137741 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.137738    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04738a95-2480-47cd-adfd-ce3e954f9156-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.138005 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.137762    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.138005 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.137809    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.138005 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.137941    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.138175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.138008    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2w2z\" (UniqueName: \"kubernetes.io/projected/04738a95-2480-47cd-adfd-ce3e954f9156-kube-api-access-l2w2z\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.138175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.138139    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.138175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.138150    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.138315 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.138295    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.140086 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.140065    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04738a95-2480-47cd-adfd-ce3e954f9156-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.140341 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.140325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04738a95-2480-47cd-adfd-ce3e954f9156-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.147329 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.147304    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2w2z\" (UniqueName: \"kubernetes.io/projected/04738a95-2480-47cd-adfd-ce3e954f9156-kube-api-access-l2w2z\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5\" (UID: \"04738a95-2480-47cd-adfd-ce3e954f9156\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.345089 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.344992    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:49.629986 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:49.629958    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"]
Sep 10 21:08:49.631228 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:08:49.631203    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04738a95_2480_47cd_adfd_ce3e954f9156.slice/crio-213d2b5059fe9617abf0f2eb3d571581d14dcd61cdb52f8ecea9f14d79b9748a WatchSource:0}: Error finding container 213d2b5059fe9617abf0f2eb3d571581d14dcd61cdb52f8ecea9f14d79b9748a: Status 404 returned error can't find the container with id 213d2b5059fe9617abf0f2eb3d571581d14dcd61cdb52f8ecea9f14d79b9748a
Sep 10 21:08:50.637343 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:50.637307    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5" event={"ID":"04738a95-2480-47cd-adfd-ce3e954f9156","Type":"ContainerStarted","Data":"33f5f7f5cde88870b891d1019622b587ed2de844591cf835d303e0de4f1b7c52"}
Sep 10 21:08:50.637849 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:50.637354    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5" event={"ID":"04738a95-2480-47cd-adfd-ce3e954f9156","Type":"ContainerStarted","Data":"213d2b5059fe9617abf0f2eb3d571581d14dcd61cdb52f8ecea9f14d79b9748a"}
Sep 10 21:08:50.660007 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:50.659957    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5" podStartSLOduration=1.469918174 podStartE2EDuration="1.659941283s" totalImagesPullingTime="190.023109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:08:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:08:50.659182118 +0000 UTC m=+610.678587419" watchObservedRunningTime="2026-09-10 21:08:50.659941283 +0000 UTC m=+610.679346641"
Sep 10 21:08:50.745334 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:50.745306    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:08:59.345397 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:59.345360    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:59.345890 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:59.345416    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:59.357526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:59.357505    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:08:59.692142 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:08:59.692057    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5"
Sep 10 21:09:42.186806 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.186769    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b76ccd69c-zrxgn"]
Sep 10 21:09:42.190216 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.190195    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.192965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.192947    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6hnqb\""
Sep 10 21:09:42.193850 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.193833    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 21:09:42.200795 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.200774    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b76ccd69c-zrxgn"]
Sep 10 21:09:42.276110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.276077    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3f63195-83b8-4fbe-91e6-afbce0e443be-tls-cert\") pod \"authorino-7b76ccd69c-zrxgn\" (UID: \"e3f63195-83b8-4fbe-91e6-afbce0e443be\") " pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.276260 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.276134    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mmcx\" (UniqueName: \"kubernetes.io/projected/e3f63195-83b8-4fbe-91e6-afbce0e443be-kube-api-access-4mmcx\") pod \"authorino-7b76ccd69c-zrxgn\" (UID: \"e3f63195-83b8-4fbe-91e6-afbce0e443be\") " pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.376822 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.376794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3f63195-83b8-4fbe-91e6-afbce0e443be-tls-cert\") pod \"authorino-7b76ccd69c-zrxgn\" (UID: \"e3f63195-83b8-4fbe-91e6-afbce0e443be\") " pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.376971 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.376850    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mmcx\" (UniqueName: \"kubernetes.io/projected/e3f63195-83b8-4fbe-91e6-afbce0e443be-kube-api-access-4mmcx\") pod \"authorino-7b76ccd69c-zrxgn\" (UID: \"e3f63195-83b8-4fbe-91e6-afbce0e443be\") " pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.379230 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.379209    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e3f63195-83b8-4fbe-91e6-afbce0e443be-tls-cert\") pod \"authorino-7b76ccd69c-zrxgn\" (UID: \"e3f63195-83b8-4fbe-91e6-afbce0e443be\") " pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.386045 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.386023    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mmcx\" (UniqueName: \"kubernetes.io/projected/e3f63195-83b8-4fbe-91e6-afbce0e443be-kube-api-access-4mmcx\") pod \"authorino-7b76ccd69c-zrxgn\" (UID: \"e3f63195-83b8-4fbe-91e6-afbce0e443be\") " pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.500278 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.500245    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b76ccd69c-zrxgn"
Sep 10 21:09:42.768553 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.768528    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b76ccd69c-zrxgn"]
Sep 10 21:09:42.770170 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:09:42.770144    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3f63195_83b8_4fbe_91e6_afbce0e443be.slice/crio-43b70f9ab1f7a65ef109c50c46d7138059b39f9bd439e4f73dc809354f2449cd WatchSource:0}: Error finding container 43b70f9ab1f7a65ef109c50c46d7138059b39f9bd439e4f73dc809354f2449cd: Status 404 returned error can't find the container with id 43b70f9ab1f7a65ef109c50c46d7138059b39f9bd439e4f73dc809354f2449cd
Sep 10 21:09:42.897582 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:42.897545    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b76ccd69c-zrxgn" event={"ID":"e3f63195-83b8-4fbe-91e6-afbce0e443be","Type":"ContainerStarted","Data":"43b70f9ab1f7a65ef109c50c46d7138059b39f9bd439e4f73dc809354f2449cd"}
Sep 10 21:09:43.904230 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:43.904191    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b76ccd69c-zrxgn" event={"ID":"e3f63195-83b8-4fbe-91e6-afbce0e443be","Type":"ContainerStarted","Data":"9ee3650311bbb50883e79e545b4c1d05dfd6de43595696db5c925591393e4bf0"}
Sep 10 21:09:43.926190 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:09:43.926132    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b76ccd69c-zrxgn" podStartSLOduration=1.404458965 podStartE2EDuration="1.926111076s" totalImagesPullingTime="521.652111ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:09:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:09:43.922484169 +0000 UTC m=+663.941889488" watchObservedRunningTime="2026-09-10 21:09:43.926111076 +0000 UTC m=+663.945516394"
Sep 10 21:10:02.453349 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.453310    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:10:02.632521 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.632488    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7644-efaf15"]
Sep 10 21:10:02.636154 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.636137    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:02.638891 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.638868    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-krpxc\""
Sep 10 21:10:02.642615 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.642595    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7644-efaf15"]
Sep 10 21:10:02.752367 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.752324    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fz5t\" (UniqueName: \"kubernetes.io/projected/64ecb9b1-e8bc-4e66-95c6-33e648c3135c-kube-api-access-4fz5t\") pod \"test-curl-7644-efaf15\" (UID: \"64ecb9b1-e8bc-4e66-95c6-33e648c3135c\") " pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:02.853298 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.853257    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fz5t\" (UniqueName: \"kubernetes.io/projected/64ecb9b1-e8bc-4e66-95c6-33e648c3135c-kube-api-access-4fz5t\") pod \"test-curl-7644-efaf15\" (UID: \"64ecb9b1-e8bc-4e66-95c6-33e648c3135c\") " pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:02.862334 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.862306    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fz5t\" (UniqueName: \"kubernetes.io/projected/64ecb9b1-e8bc-4e66-95c6-33e648c3135c-kube-api-access-4fz5t\") pod \"test-curl-7644-efaf15\" (UID: \"64ecb9b1-e8bc-4e66-95c6-33e648c3135c\") " pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:02.947032 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:02.946995    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:03.216739 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:03.216709    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7644-efaf15"]
Sep 10 21:10:03.218411 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:03.218377    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64ecb9b1_e8bc_4e66_95c6_33e648c3135c.slice/crio-e6f0da584a418a47aee6fc2aca5f965e5c6da15716cbed853e788623e741f92f WatchSource:0}: Error finding container e6f0da584a418a47aee6fc2aca5f965e5c6da15716cbed853e788623e741f92f: Status 404 returned error can't find the container with id e6f0da584a418a47aee6fc2aca5f965e5c6da15716cbed853e788623e741f92f
Sep 10 21:10:03.220382 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:03.220366    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:10:04.015115 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:04.015080    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7644-efaf15" event={"ID":"64ecb9b1-e8bc-4e66-95c6-33e648c3135c","Type":"ContainerStarted","Data":"e6f0da584a418a47aee6fc2aca5f965e5c6da15716cbed853e788623e741f92f"}
Sep 10 21:10:07.033176 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.033128    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7644-efaf15" event={"ID":"64ecb9b1-e8bc-4e66-95c6-33e648c3135c","Type":"ContainerStarted","Data":"9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45"}
Sep 10 21:10:07.049699 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.049638    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7644-efaf15" podStartSLOduration=1.848419331 podStartE2EDuration="5.049621653s" totalImagesPullingTime="3.201202322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:07.047348805 +0000 UTC m=+687.066754124" watchObservedRunningTime="2026-09-10 21:10:07.049621653 +0000 UTC m=+687.069027028"
Sep 10 21:10:07.613832 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.613794    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7644-efaf15"]
Sep 10 21:10:07.620607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.620580    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7644-efaf15"]
Sep 10 21:10:07.736002 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.735960    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7644-17cf25"]
Sep 10 21:10:07.736332 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.736316    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ecb9b1-e8bc-4e66-95c6-33e648c3135c" containerName="test-curl-7644-efaf15"
Sep 10 21:10:07.736332 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.736330    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ecb9b1-e8bc-4e66-95c6-33e648c3135c" containerName="test-curl-7644-efaf15"
Sep 10 21:10:07.736496 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.736397    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ecb9b1-e8bc-4e66-95c6-33e648c3135c" containerName="test-curl-7644-efaf15"
Sep 10 21:10:07.739634 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.739608    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:07.744459 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.744435    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7644-17cf25"]
Sep 10 21:10:07.801219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.801183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fmhq\" (UniqueName: \"kubernetes.io/projected/9a97844c-d9a6-485a-843a-1d9788648dcb-kube-api-access-9fmhq\") pod \"test-curl-7644-17cf25\" (UID: \"9a97844c-d9a6-485a-843a-1d9788648dcb\") " pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:07.902173 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.902137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fmhq\" (UniqueName: \"kubernetes.io/projected/9a97844c-d9a6-485a-843a-1d9788648dcb-kube-api-access-9fmhq\") pod \"test-curl-7644-17cf25\" (UID: \"9a97844c-d9a6-485a-843a-1d9788648dcb\") " pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:07.910530 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:07.910503    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fmhq\" (UniqueName: \"kubernetes.io/projected/9a97844c-d9a6-485a-843a-1d9788648dcb-kube-api-access-9fmhq\") pod \"test-curl-7644-17cf25\" (UID: \"9a97844c-d9a6-485a-843a-1d9788648dcb\") " pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:08.039611 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:08.039528    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7644-efaf15" container="test-curl-7644-efaf15"
Sep 10 21:10:08.050766 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:08.050742    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:08.342244 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:08.342214    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7644-17cf25"]
Sep 10 21:10:08.344101 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:08.344068    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a97844c_d9a6_485a_843a_1d9788648dcb.slice/crio-f8a6673b8937b869909f9c8413c31953156f9b7788f4aecc3c1836b61ea01557 WatchSource:0}: Error finding container f8a6673b8937b869909f9c8413c31953156f9b7788f4aecc3c1836b61ea01557: Status 404 returned error can't find the container with id f8a6673b8937b869909f9c8413c31953156f9b7788f4aecc3c1836b61ea01557
Sep 10 21:10:09.045772 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:09.045733    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7644-17cf25" event={"ID":"9a97844c-d9a6-485a-843a-1d9788648dcb","Type":"ContainerStarted","Data":"189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689"}
Sep 10 21:10:09.046160 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:09.045783    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7644-17cf25" event={"ID":"9a97844c-d9a6-485a-843a-1d9788648dcb","Type":"ContainerStarted","Data":"f8a6673b8937b869909f9c8413c31953156f9b7788f4aecc3c1836b61ea01557"}
Sep 10 21:10:09.046160 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:09.045828    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7644-efaf15" podUID="64ecb9b1-e8bc-4e66-95c6-33e648c3135c" containerName="test-curl-7644-efaf15" containerID="cri-o://9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45" gracePeriod=2
Sep 10 21:10:09.061669 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:09.061625    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7644-17cf25" podStartSLOduration=2.061610164 podStartE2EDuration="2.061610164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:09.059540569 +0000 UTC m=+689.078945887" watchObservedRunningTime="2026-09-10 21:10:09.061610164 +0000 UTC m=+689.081015479"
Sep 10 21:10:09.652416 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:09.652383    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7644-17cf25"]
Sep 10 21:10:09.658707 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:09.658652    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7644-17cf25"]
Sep 10 21:10:10.051276 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:10.051242    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7644-17cf25" container="test-curl-7644-17cf25"
Sep 10 21:10:11.057022 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:11.056983    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7644-17cf25" podUID="9a97844c-d9a6-485a-843a-1d9788648dcb" containerName="test-curl-7644-17cf25" containerID="cri-o://189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689" gracePeriod=2
Sep 10 21:10:11.227877 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:11.227854    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:11.335358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:11.335278    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fz5t\" (UniqueName: \"kubernetes.io/projected/64ecb9b1-e8bc-4e66-95c6-33e648c3135c-kube-api-access-4fz5t\") pod \"64ecb9b1-e8bc-4e66-95c6-33e648c3135c\" (UID: \"64ecb9b1-e8bc-4e66-95c6-33e648c3135c\") "
Sep 10 21:10:11.337372 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:11.337344    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64ecb9b1-e8bc-4e66-95c6-33e648c3135c-kube-api-access-4fz5t" (OuterVolumeSpecName: "kube-api-access-4fz5t") pod "64ecb9b1-e8bc-4e66-95c6-33e648c3135c" (UID: "64ecb9b1-e8bc-4e66-95c6-33e648c3135c"). InnerVolumeSpecName "kube-api-access-4fz5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:10:11.436377 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:11.436345    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fz5t\" (UniqueName: \"kubernetes.io/projected/64ecb9b1-e8bc-4e66-95c6-33e648c3135c-kube-api-access-4fz5t\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:12.063802 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.063770    2557 generic.go:356] "Generic (PLEG): container finished" podID="64ecb9b1-e8bc-4e66-95c6-33e648c3135c" containerID="9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45" exitCode=137
Sep 10 21:10:12.064192 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.063813    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:12.064192 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.063879    2557 scope.go:117] "RemoveContainer" containerID="9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45"
Sep 10 21:10:12.073174 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.073154    2557 scope.go:117] "RemoveContainer" containerID="9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45"
Sep 10 21:10:12.073431 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:12.073410    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45\": container with ID starting with 9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45 not found: ID does not exist" containerID="9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45"
Sep 10 21:10:12.073490 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.073443    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45"} err="failed to get container status \"9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45\": rpc error: code = NotFound desc = could not find container \"9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45\": container with ID starting with 9bd4598841db5a927a564c1703e5085e6f79f18f8368b1288d56a1265672da45 not found: ID does not exist"
Sep 10 21:10:12.074094 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:12.074072    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7644-efaf15\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="64ecb9b1-e8bc-4e66-95c6-33e648c3135c" pod="opendatahub/test-curl-7644-efaf15"
Sep 10 21:10:12.585302 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.585261    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64ecb9b1-e8bc-4e66-95c6-33e648c3135c" path="/var/lib/kubelet/pods/64ecb9b1-e8bc-4e66-95c6-33e648c3135c/volumes"
Sep 10 21:10:12.935143 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.935036    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"]
Sep 10 21:10:12.935882 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.935860    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a97844c-d9a6-485a-843a-1d9788648dcb" containerName="test-curl-7644-17cf25"
Sep 10 21:10:12.936006 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.935995    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a97844c-d9a6-485a-843a-1d9788648dcb" containerName="test-curl-7644-17cf25"
Sep 10 21:10:12.936186 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.936175    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a97844c-d9a6-485a-843a-1d9788648dcb" containerName="test-curl-7644-17cf25"
Sep 10 21:10:12.944938 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.944911    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:12.947641 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.947619    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:10:12.949579 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.949550    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-d9fd5e6e-serving-cert\""
Sep 10 21:10:12.949773 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.949747    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:10:12.950090 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.950072    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:10:12.950219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.950113    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:10:12.954607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:12.954588    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"]
Sep 10 21:10:13.050523 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.050495    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d521272-5da7-46e5-aaee-9e28638bc053-openshift-service-ca\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.050663 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.050535    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-metrics-certs\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.050663 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.050558    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfsd9\" (UniqueName: \"kubernetes.io/projected/8d521272-5da7-46e5-aaee-9e28638bc053-kube-api-access-lfsd9\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.050663 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.050651    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-maas-api-tls\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.151816 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.151780    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-maas-api-tls\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.152226 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.151872    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d521272-5da7-46e5-aaee-9e28638bc053-openshift-service-ca\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.152226 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.151907    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-metrics-certs\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.152226 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.151932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfsd9\" (UniqueName: \"kubernetes.io/projected/8d521272-5da7-46e5-aaee-9e28638bc053-kube-api-access-lfsd9\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.152563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.152536    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d521272-5da7-46e5-aaee-9e28638bc053-openshift-service-ca\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.154179 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.154154    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-metrics-certs\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.154329 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.154311    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-maas-api-tls\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.167268 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.167243    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfsd9\" (UniqueName: \"kubernetes.io/projected/8d521272-5da7-46e5-aaee-9e28638bc053-kube-api-access-lfsd9\") pod \"maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.240342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.240318    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:13.260523 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.260496    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:13.352923 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.352893    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fmhq\" (UniqueName: \"kubernetes.io/projected/9a97844c-d9a6-485a-843a-1d9788648dcb-kube-api-access-9fmhq\") pod \"9a97844c-d9a6-485a-843a-1d9788648dcb\" (UID: \"9a97844c-d9a6-485a-843a-1d9788648dcb\") "
Sep 10 21:10:13.354722 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.354673    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a97844c-d9a6-485a-843a-1d9788648dcb-kube-api-access-9fmhq" (OuterVolumeSpecName: "kube-api-access-9fmhq") pod "9a97844c-d9a6-485a-843a-1d9788648dcb" (UID: "9a97844c-d9a6-485a-843a-1d9788648dcb"). InnerVolumeSpecName "kube-api-access-9fmhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:10:13.454326 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.454282    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fmhq\" (UniqueName: \"kubernetes.io/projected/9a97844c-d9a6-485a-843a-1d9788648dcb-kube-api-access-9fmhq\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:13.531741 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.531715    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"]
Sep 10 21:10:13.533310 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:13.533275    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d521272_5da7_46e5_aaee_9e28638bc053.slice/crio-bacd37927826a18a98b30515e7b647f27e1d02ca2c2bbe2f67957923c20a61af WatchSource:0}: Error finding container bacd37927826a18a98b30515e7b647f27e1d02ca2c2bbe2f67957923c20a61af: Status 404 returned error can't find the container with id bacd37927826a18a98b30515e7b647f27e1d02ca2c2bbe2f67957923c20a61af
Sep 10 21:10:13.746676 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.746637    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"]
Sep 10 21:10:13.752079 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.752053    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:13.754654 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.754628    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d9fd5e6e\""
Sep 10 21:10:13.754869 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.754854    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d9fd5e6e-dockercfg-rt65r\""
Sep 10 21:10:13.758467 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.758444    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"]
Sep 10 21:10:13.857758 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.857670    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st4c9\" (UniqueName: \"kubernetes.io/projected/e94385b2-63ec-43e2-afb7-7503ac5a8b09-kube-api-access-st4c9\") pod \"payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") " pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:13.857758 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.857728    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94385b2-63ec-43e2-afb7-7503ac5a8b09-plugins-config-volume\") pod \"payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") " pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:13.958571 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.958531    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st4c9\" (UniqueName: \"kubernetes.io/projected/e94385b2-63ec-43e2-afb7-7503ac5a8b09-kube-api-access-st4c9\") pod \"payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") " pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:13.958753 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.958586    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94385b2-63ec-43e2-afb7-7503ac5a8b09-plugins-config-volume\") pod \"payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") " pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:13.959485 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.959460    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94385b2-63ec-43e2-afb7-7503ac5a8b09-plugins-config-volume\") pod \"payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") " pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:13.969809 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:13.969777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st4c9\" (UniqueName: \"kubernetes.io/projected/e94385b2-63ec-43e2-afb7-7503ac5a8b09-kube-api-access-st4c9\") pod \"payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") " pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:14.063090 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.063057    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:14.084718 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.084596    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6" event={"ID":"8d521272-5da7-46e5-aaee-9e28638bc053","Type":"ContainerStarted","Data":"bacd37927826a18a98b30515e7b647f27e1d02ca2c2bbe2f67957923c20a61af"}
Sep 10 21:10:14.087093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.087051    2557 generic.go:356] "Generic (PLEG): container finished" podID="9a97844c-d9a6-485a-843a-1d9788648dcb" containerID="189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689" exitCode=137
Sep 10 21:10:14.087219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.087099    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:14.087219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.087112    2557 scope.go:117] "RemoveContainer" containerID="189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689"
Sep 10 21:10:14.104367 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:14.104332    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7644-17cf25\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="9a97844c-d9a6-485a-843a-1d9788648dcb" pod="opendatahub/test-curl-7644-17cf25"
Sep 10 21:10:14.105268 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.105065    2557 scope.go:117] "RemoveContainer" containerID="189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689"
Sep 10 21:10:14.105448 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:14.105421    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689\": container with ID starting with 189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689 not found: ID does not exist" containerID="189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689"
Sep 10 21:10:14.105512 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.105461    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689"} err="failed to get container status \"189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689\": rpc error: code = NotFound desc = could not find container \"189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689\": container with ID starting with 189d6dfb3ef9c100212dfce7c872e489fe6cc40de8e2b70d0e92cb29b4897689 not found: ID does not exist"
Sep 10 21:10:14.362863 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.362830    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"]
Sep 10 21:10:14.367206 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:14.367164    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode94385b2_63ec_43e2_afb7_7503ac5a8b09.slice/crio-d1ab86b0ef56a7e4eaa623bfaf112227b310581cf6c089b72288c0e7ada9e441 WatchSource:0}: Error finding container d1ab86b0ef56a7e4eaa623bfaf112227b310581cf6c089b72288c0e7ada9e441: Status 404 returned error can't find the container with id d1ab86b0ef56a7e4eaa623bfaf112227b310581cf6c089b72288c0e7ada9e441
Sep 10 21:10:14.587171 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:14.587138    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a97844c-d9a6-485a-843a-1d9788648dcb" path="/var/lib/kubelet/pods/9a97844c-d9a6-485a-843a-1d9788648dcb/volumes"
Sep 10 21:10:15.098337 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.098296    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j" event={"ID":"e94385b2-63ec-43e2-afb7-7503ac5a8b09","Type":"ContainerStarted","Data":"3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9"}
Sep 10 21:10:15.098553 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.098362    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j" event={"ID":"e94385b2-63ec-43e2-afb7-7503ac5a8b09","Type":"ContainerStarted","Data":"d1ab86b0ef56a7e4eaa623bfaf112227b310581cf6c089b72288c0e7ada9e441"}
Sep 10 21:10:15.098553 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.098414    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:15.140965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.140906    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j" podStartSLOduration=2.140889258 podStartE2EDuration="2.140889258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:15.134798883 +0000 UTC m=+695.154204201" watchObservedRunningTime="2026-09-10 21:10:15.140889258 +0000 UTC m=+695.160294577"
Sep 10 21:10:15.155897 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.155858    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:10:15.474536 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.474502    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"]
Sep 10 21:10:15.478455 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.478431    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.480963 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.480938    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-f96a7a\""
Sep 10 21:10:15.496440 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.496414    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"]
Sep 10 21:10:15.506832 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.506804    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"]
Sep 10 21:10:15.510563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.510541    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:15.513458 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.513429    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-f96a7a-dockercfg-d42hk\""
Sep 10 21:10:15.522298 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.522273    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"]
Sep 10 21:10:15.575619 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.575576    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d4c5e5-467f-4678-802d-5d500e1ec283-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.582861 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.578807    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2c8h\" (UniqueName: \"kubernetes.io/projected/c9d4c5e5-467f-4678-802d-5d500e1ec283-kube-api-access-c2c8h\") pod \"payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.679982 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.679941    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-plugins-config-volume\") pod \"payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") " pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:15.680159 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.680063    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d4c5e5-467f-4678-802d-5d500e1ec283-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.680159 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.680137    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59865\" (UniqueName: \"kubernetes.io/projected/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-kube-api-access-59865\") pod \"payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") " pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:15.680282 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.680168    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2c8h\" (UniqueName: \"kubernetes.io/projected/c9d4c5e5-467f-4678-802d-5d500e1ec283-kube-api-access-c2c8h\") pod \"payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.680837 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.680812    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d4c5e5-467f-4678-802d-5d500e1ec283-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.689766 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.689723    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2c8h\" (UniqueName: \"kubernetes.io/projected/c9d4c5e5-467f-4678-802d-5d500e1ec283-kube-api-access-c2c8h\") pod \"payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.781738 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.781617    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59865\" (UniqueName: \"kubernetes.io/projected/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-kube-api-access-59865\") pod \"payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") " pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:15.781901 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.781742    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-plugins-config-volume\") pod \"payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") " pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:15.782726 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.782657    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-plugins-config-volume\") pod \"payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") " pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:15.791377 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.791349    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59865\" (UniqueName: \"kubernetes.io/projected/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-kube-api-access-59865\") pod \"payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") " pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:15.794153 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.794089    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:15.825225 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:15.825196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:16.077267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:16.077236    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"]
Sep 10 21:10:16.078781 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:16.078750    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9d4c5e5_467f_4678_802d_5d500e1ec283.slice/crio-f06f1c896ea9eb346ee6b58e56c474829598ba8e5cd891f235b4630fbaa26e16 WatchSource:0}: Error finding container f06f1c896ea9eb346ee6b58e56c474829598ba8e5cd891f235b4630fbaa26e16: Status 404 returned error can't find the container with id f06f1c896ea9eb346ee6b58e56c474829598ba8e5cd891f235b4630fbaa26e16
Sep 10 21:10:16.106398 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:16.106366    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98" event={"ID":"c9d4c5e5-467f-4678-802d-5d500e1ec283","Type":"ContainerStarted","Data":"f06f1c896ea9eb346ee6b58e56c474829598ba8e5cd891f235b4630fbaa26e16"}
Sep 10 21:10:16.107984 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:16.107959    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6" event={"ID":"8d521272-5da7-46e5-aaee-9e28638bc053","Type":"ContainerStarted","Data":"fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86"}
Sep 10 21:10:16.108076 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:16.108027    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:16.128968 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:16.128924    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6" podStartSLOduration=1.791920378 podStartE2EDuration="4.128905154s" totalImagesPullingTime="2.336984776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:16.127588386 +0000 UTC m=+696.146993704" watchObservedRunningTime="2026-09-10 21:10:16.128905154 +0000 UTC m=+696.148310492"
Sep 10 21:10:16.144321 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:16.144298    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"]
Sep 10 21:10:16.145831 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:16.145783    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72634e55_dc83_4bc7_9185_d3fc4f50bb9c.slice/crio-8e65a74f97b379fcc6ad978ff450f1bb4403df22cb49114a9478fe9610293052 WatchSource:0}: Error finding container 8e65a74f97b379fcc6ad978ff450f1bb4403df22cb49114a9478fe9610293052: Status 404 returned error can't find the container with id 8e65a74f97b379fcc6ad978ff450f1bb4403df22cb49114a9478fe9610293052
Sep 10 21:10:17.114879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:17.114839    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4" event={"ID":"72634e55-dc83-4bc7-9185-d3fc4f50bb9c","Type":"ContainerStarted","Data":"b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649"}
Sep 10 21:10:17.115331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:17.114904    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4" event={"ID":"72634e55-dc83-4bc7-9185-d3fc4f50bb9c","Type":"ContainerStarted","Data":"8e65a74f97b379fcc6ad978ff450f1bb4403df22cb49114a9478fe9610293052"}
Sep 10 21:10:17.115331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:17.114938    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:17.116488 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:17.116465    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98" event={"ID":"c9d4c5e5-467f-4678-802d-5d500e1ec283","Type":"ContainerStarted","Data":"8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38"}
Sep 10 21:10:17.116599 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:17.116505    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:17.142844 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:17.142800    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4" podStartSLOduration=2.142785506 podStartE2EDuration="2.142785506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:17.140481389 +0000 UTC m=+697.159886707" watchObservedRunningTime="2026-09-10 21:10:17.142785506 +0000 UTC m=+697.162190824"
Sep 10 21:10:17.165796 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:17.165746    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98" podStartSLOduration=2.165730534 podStartE2EDuration="2.165730534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:17.162530496 +0000 UTC m=+697.181935902" watchObservedRunningTime="2026-09-10 21:10:17.165730534 +0000 UTC m=+697.185135852"
Sep 10 21:10:22.122483 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:22.122453    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:26.110162 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:26.110135    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:28.123532 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:28.123497    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:10:28.123911 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:28.123579    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:10:32.860612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:32.860525    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:10:34.138414 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.138382    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"]
Sep 10 21:10:34.142792 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.142768    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.145925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.145904    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-31e7cb-dockercfg-wsf29\""
Sep 10 21:10:34.146025 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.146000    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-31e7cb\""
Sep 10 21:10:34.156706 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.156662    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"]
Sep 10 21:10:34.246612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.246580    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/757237a7-e86c-4890-b909-ed7791d794d4-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.246787 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.246639    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlrbm\" (UniqueName: \"kubernetes.io/projected/757237a7-e86c-4890-b909-ed7791d794d4-kube-api-access-tlrbm\") pod \"payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.347342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.347305    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/757237a7-e86c-4890-b909-ed7791d794d4-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.347519 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.347365    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlrbm\" (UniqueName: \"kubernetes.io/projected/757237a7-e86c-4890-b909-ed7791d794d4-kube-api-access-tlrbm\") pod \"payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.348052 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.348032    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/757237a7-e86c-4890-b909-ed7791d794d4-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.357707 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.357647    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlrbm\" (UniqueName: \"kubernetes.io/projected/757237a7-e86c-4890-b909-ed7791d794d4-kube-api-access-tlrbm\") pod \"payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") " pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.453556 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.453470    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:34.728512 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:34.728486    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"]
Sep 10 21:10:34.730058 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:34.730032    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod757237a7_e86c_4890_b909_ed7791d794d4.slice/crio-26a165092397e9ee4344929d418c81d3c33ceca2f8754312cd65b4fe80fe470a WatchSource:0}: Error finding container 26a165092397e9ee4344929d418c81d3c33ceca2f8754312cd65b4fe80fe470a: Status 404 returned error can't find the container with id 26a165092397e9ee4344929d418c81d3c33ceca2f8754312cd65b4fe80fe470a
Sep 10 21:10:35.216159 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:35.216120    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l" event={"ID":"757237a7-e86c-4890-b909-ed7791d794d4","Type":"ContainerStarted","Data":"628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c"}
Sep 10 21:10:35.216550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:35.216185    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l" event={"ID":"757237a7-e86c-4890-b909-ed7791d794d4","Type":"ContainerStarted","Data":"26a165092397e9ee4344929d418c81d3c33ceca2f8754312cd65b4fe80fe470a"}
Sep 10 21:10:35.216550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:35.216213    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:35.239421 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:35.239365    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l" podStartSLOduration=1.2393511130000001 podStartE2EDuration="1.239351113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:10:35.236043357 +0000 UTC m=+715.255448686" watchObservedRunningTime="2026-09-10 21:10:35.239351113 +0000 UTC m=+715.258756424"
Sep 10 21:10:46.224035 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:46.224004    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:10:50.658971 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:50.658943    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:10:53.755780 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:53.755741    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"]
Sep 10 21:10:53.756846 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:53.756742    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6" podUID="8d521272-5da7-46e5-aaee-9e28638bc053" containerName="maas-api" containerID="cri-o://fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86" gracePeriod=30
Sep 10 21:10:53.809965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:53.809932    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"]
Sep 10 21:10:53.810169 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:53.810130    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j" podUID="e94385b2-63ec-43e2-afb7-7503ac5a8b09" containerName="payload-processing" containerID="cri-o://3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9" gracePeriod=30
Sep 10 21:10:54.049206 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.049179    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:54.077151 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.077120    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:54.125101 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.125075    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d521272-5da7-46e5-aaee-9e28638bc053-openshift-service-ca\") pod \"8d521272-5da7-46e5-aaee-9e28638bc053\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") "
Sep 10 21:10:54.125253 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.125178    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfsd9\" (UniqueName: \"kubernetes.io/projected/8d521272-5da7-46e5-aaee-9e28638bc053-kube-api-access-lfsd9\") pod \"8d521272-5da7-46e5-aaee-9e28638bc053\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") "
Sep 10 21:10:54.125253 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.125205    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-maas-api-tls\") pod \"8d521272-5da7-46e5-aaee-9e28638bc053\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") "
Sep 10 21:10:54.125253 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.125249    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-metrics-certs\") pod \"8d521272-5da7-46e5-aaee-9e28638bc053\" (UID: \"8d521272-5da7-46e5-aaee-9e28638bc053\") "
Sep 10 21:10:54.125504 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.125472    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d521272-5da7-46e5-aaee-9e28638bc053-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8d521272-5da7-46e5-aaee-9e28638bc053" (UID: "8d521272-5da7-46e5-aaee-9e28638bc053"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:10:54.127424 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.127391    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8d521272-5da7-46e5-aaee-9e28638bc053" (UID: "8d521272-5da7-46e5-aaee-9e28638bc053"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:10:54.127541 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.127424    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8d521272-5da7-46e5-aaee-9e28638bc053" (UID: "8d521272-5da7-46e5-aaee-9e28638bc053"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:10:54.127541 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.127433    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d521272-5da7-46e5-aaee-9e28638bc053-kube-api-access-lfsd9" (OuterVolumeSpecName: "kube-api-access-lfsd9") pod "8d521272-5da7-46e5-aaee-9e28638bc053" (UID: "8d521272-5da7-46e5-aaee-9e28638bc053"). InnerVolumeSpecName "kube-api-access-lfsd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:10:54.226216 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.226183    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-st4c9\" (UniqueName: \"kubernetes.io/projected/e94385b2-63ec-43e2-afb7-7503ac5a8b09-kube-api-access-st4c9\") pod \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") "
Sep 10 21:10:54.226384 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.226232    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94385b2-63ec-43e2-afb7-7503ac5a8b09-plugins-config-volume\") pod \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\" (UID: \"e94385b2-63ec-43e2-afb7-7503ac5a8b09\") "
Sep 10 21:10:54.226590 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.226570    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfsd9\" (UniqueName: \"kubernetes.io/projected/8d521272-5da7-46e5-aaee-9e28638bc053-kube-api-access-lfsd9\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.226659 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.226595    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.226659 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.226610    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d521272-5da7-46e5-aaee-9e28638bc053-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.226659 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.226609    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94385b2-63ec-43e2-afb7-7503ac5a8b09-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e94385b2-63ec-43e2-afb7-7503ac5a8b09" (UID: "e94385b2-63ec-43e2-afb7-7503ac5a8b09"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:10:54.226659 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.226624    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d521272-5da7-46e5-aaee-9e28638bc053-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.228385 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.228366    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94385b2-63ec-43e2-afb7-7503ac5a8b09-kube-api-access-st4c9" (OuterVolumeSpecName: "kube-api-access-st4c9") pod "e94385b2-63ec-43e2-afb7-7503ac5a8b09" (UID: "e94385b2-63ec-43e2-afb7-7503ac5a8b09"). InnerVolumeSpecName "kube-api-access-st4c9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:10:54.323888 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.323802    2557 generic.go:356] "Generic (PLEG): container finished" podID="8d521272-5da7-46e5-aaee-9e28638bc053" containerID="fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86" exitCode=0
Sep 10 21:10:54.323888 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.323851    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"
Sep 10 21:10:54.324064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.323885    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6" event={"ID":"8d521272-5da7-46e5-aaee-9e28638bc053","Type":"ContainerDied","Data":"fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86"}
Sep 10 21:10:54.324064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.323945    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6" event={"ID":"8d521272-5da7-46e5-aaee-9e28638bc053","Type":"ContainerDied","Data":"bacd37927826a18a98b30515e7b647f27e1d02ca2c2bbe2f67957923c20a61af"}
Sep 10 21:10:54.324064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.323969    2557 scope.go:117] "RemoveContainer" containerID="fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86"
Sep 10 21:10:54.325347 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.325322    2557 generic.go:356] "Generic (PLEG): container finished" podID="e94385b2-63ec-43e2-afb7-7503ac5a8b09" containerID="3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9" exitCode=0
Sep 10 21:10:54.325454 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.325376    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"
Sep 10 21:10:54.325454 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.325406    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j" event={"ID":"e94385b2-63ec-43e2-afb7-7503ac5a8b09","Type":"ContainerDied","Data":"3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9"}
Sep 10 21:10:54.325558 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.325458    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j" event={"ID":"e94385b2-63ec-43e2-afb7-7503ac5a8b09","Type":"ContainerDied","Data":"d1ab86b0ef56a7e4eaa623bfaf112227b310581cf6c089b72288c0e7ada9e441"}
Sep 10 21:10:54.327331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.327311    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st4c9\" (UniqueName: \"kubernetes.io/projected/e94385b2-63ec-43e2-afb7-7503ac5a8b09-kube-api-access-st4c9\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.327432 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.327334    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94385b2-63ec-43e2-afb7-7503ac5a8b09-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:10:54.335043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.335022    2557 scope.go:117] "RemoveContainer" containerID="fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86"
Sep 10 21:10:54.335367 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:54.335345    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86\": container with ID starting with fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86 not found: ID does not exist" containerID="fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86"
Sep 10 21:10:54.335468 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.335373    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86"} err="failed to get container status \"fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86\": rpc error: code = NotFound desc = could not find container \"fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86\": container with ID starting with fcd4ac0f01386b7f359ac91a6209201229a311e5e79ec13995458bc3962daa86 not found: ID does not exist"
Sep 10 21:10:54.335468 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.335396    2557 scope.go:117] "RemoveContainer" containerID="3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9"
Sep 10 21:10:54.344285 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.344266    2557 scope.go:117] "RemoveContainer" containerID="3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9"
Sep 10 21:10:54.344551 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:54.344537    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9\": container with ID starting with 3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9 not found: ID does not exist" containerID="3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9"
Sep 10 21:10:54.344606 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.344558    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9"} err="failed to get container status \"3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9\": rpc error: code = NotFound desc = could not find container \"3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9\": container with ID starting with 3492a34e820391456f5dc0fc66eafc7cb3bb4b384579c33d96c26e1a772ee6b9 not found: ID does not exist"
Sep 10 21:10:54.353724 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.353700    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"]
Sep 10 21:10:54.358862 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.358840    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-d9fd5e6e-6d6df6957b-dnmn6"]
Sep 10 21:10:54.372508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.372481    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"]
Sep 10 21:10:54.377655 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.377631    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-d9fd5e6e-576859c5d7-b545j"]
Sep 10 21:10:54.585774 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.585672    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d521272-5da7-46e5-aaee-9e28638bc053" path="/var/lib/kubelet/pods/8d521272-5da7-46e5-aaee-9e28638bc053/volumes"
Sep 10 21:10:54.586065 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:54.586052    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e94385b2-63ec-43e2-afb7-7503ac5a8b09" path="/var/lib/kubelet/pods/e94385b2-63ec-43e2-afb7-7503ac5a8b09/volumes"
Sep 10 21:10:58.177429 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.177389    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"]
Sep 10 21:10:58.177992 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.177973    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e94385b2-63ec-43e2-afb7-7503ac5a8b09" containerName="payload-processing"
Sep 10 21:10:58.177992 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.177993    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e94385b2-63ec-43e2-afb7-7503ac5a8b09" containerName="payload-processing"
Sep 10 21:10:58.178131 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.178019    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d521272-5da7-46e5-aaee-9e28638bc053" containerName="maas-api"
Sep 10 21:10:58.178131 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.178028    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d521272-5da7-46e5-aaee-9e28638bc053" containerName="maas-api"
Sep 10 21:10:58.178247 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.178132    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e94385b2-63ec-43e2-afb7-7503ac5a8b09" containerName="payload-processing"
Sep 10 21:10:58.178247 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.178146    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d521272-5da7-46e5-aaee-9e28638bc053" containerName="maas-api"
Sep 10 21:10:58.181195 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.181172    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.185710 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.185669    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-bbddbd71-serving-cert\""
Sep 10 21:10:58.186187 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.186165    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:10:58.187053 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.187009    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:10:58.187214 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.187192    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:10:58.187296 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.187225    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:10:58.199227 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.199200    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"]
Sep 10 21:10:58.262125 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.262092    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc215ab-e08c-46f7-a64c-7bf237683b44-openshift-service-ca\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.262285 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.262147    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.262285 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.262164    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-metrics-certs\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.262285 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.262230    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shpsv\" (UniqueName: \"kubernetes.io/projected/1bc215ab-e08c-46f7-a64c-7bf237683b44-kube-api-access-shpsv\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.363768 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.363734    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc215ab-e08c-46f7-a64c-7bf237683b44-openshift-service-ca\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.363938 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.363815    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.363938 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.363842    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-metrics-certs\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.363938 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.363874    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shpsv\" (UniqueName: \"kubernetes.io/projected/1bc215ab-e08c-46f7-a64c-7bf237683b44-kube-api-access-shpsv\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.364126 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:58.363954    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-serving-cert: secret "maas-api-e2e-migrate-bbddbd71-serving-cert" not found
Sep 10 21:10:58.364126 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:10:58.364033    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls podName:1bc215ab-e08c-46f7-a64c-7bf237683b44 nodeName:}" failed. No retries permitted until 2026-09-10 21:10:58.86401494 +0000 UTC m=+738.883420235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls") pod "maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs" (UID: "1bc215ab-e08c-46f7-a64c-7bf237683b44") : secret "maas-api-e2e-migrate-bbddbd71-serving-cert" not found
Sep 10 21:10:58.364411 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.364389    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc215ab-e08c-46f7-a64c-7bf237683b44-openshift-service-ca\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.366295 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.366276    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-metrics-certs\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.376339 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.376308    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shpsv\" (UniqueName: \"kubernetes.io/projected/1bc215ab-e08c-46f7-a64c-7bf237683b44-kube-api-access-shpsv\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.867370 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.867338    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:58.869650 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:58.869620    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls\") pod \"maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:59.092859 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.092827    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:10:59.231860 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.231828    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"]
Sep 10 21:10:59.234574 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.234550    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.237238 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.237217    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-bbddbd71-dockercfg-gz4d5\""
Sep 10 21:10:59.237384 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.237366    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-bbddbd71\""
Sep 10 21:10:59.250273 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.250250    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"]
Sep 10 21:10:59.333537 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.333503    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"]
Sep 10 21:10:59.335845 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.335830    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.352448 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.352423    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"]
Sep 10 21:10:59.372534 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.372506    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bhwr\" (UniqueName: \"kubernetes.io/projected/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-kube-api-access-4bhwr\") pod \"payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.372651 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.372537    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.380397 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.380378    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"]
Sep 10 21:10:59.382161 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:59.382137    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bc215ab_e08c_46f7_a64c_7bf237683b44.slice/crio-c875cfb315f48000b0d0b9c713fc1961ff0674215aa0393d686d0183afa829ad WatchSource:0}: Error finding container c875cfb315f48000b0d0b9c713fc1961ff0674215aa0393d686d0183afa829ad: Status 404 returned error can't find the container with id c875cfb315f48000b0d0b9c713fc1961ff0674215aa0393d686d0183afa829ad
Sep 10 21:10:59.473410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.473378    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a98e1a-65c7-4598-aedb-685f1b4519df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") " pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.473560 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.473429    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bhwr\" (UniqueName: \"kubernetes.io/projected/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-kube-api-access-4bhwr\") pod \"payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.473560 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.473455    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.473560 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.473474    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc6cv\" (UniqueName: \"kubernetes.io/projected/09a98e1a-65c7-4598-aedb-685f1b4519df-kube-api-access-kc6cv\") pod \"payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") " pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.474146 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.474126    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.483274 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.483251    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bhwr\" (UniqueName: \"kubernetes.io/projected/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-kube-api-access-4bhwr\") pod \"payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.545799 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.545766    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:10:59.574130 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.574104    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a98e1a-65c7-4598-aedb-685f1b4519df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") " pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.574250 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.574149    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc6cv\" (UniqueName: \"kubernetes.io/projected/09a98e1a-65c7-4598-aedb-685f1b4519df-kube-api-access-kc6cv\") pod \"payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") " pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.574761 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.574744    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a98e1a-65c7-4598-aedb-685f1b4519df-plugins-config-volume\") pod \"payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") " pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.584342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.584315    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc6cv\" (UniqueName: \"kubernetes.io/projected/09a98e1a-65c7-4598-aedb-685f1b4519df-kube-api-access-kc6cv\") pod \"payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") " pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.645784 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.645753    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:10:59.845828 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.845795    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"]
Sep 10 21:10:59.851009 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:59.850972    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ca8dcc6_817b_4f6b_bed5_e89cd07ab8cf.slice/crio-0878ab7dcb4c368dc144ab2d60fe751ad52645cdcf00120d1daa1f47dd7701b1 WatchSource:0}: Error finding container 0878ab7dcb4c368dc144ab2d60fe751ad52645cdcf00120d1daa1f47dd7701b1: Status 404 returned error can't find the container with id 0878ab7dcb4c368dc144ab2d60fe751ad52645cdcf00120d1daa1f47dd7701b1
Sep 10 21:10:59.929574 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:10:59.929547    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"]
Sep 10 21:10:59.931324 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:10:59.931294    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a98e1a_65c7_4598_aedb_685f1b4519df.slice/crio-d870e3123b77d64f8d05690ba6537b4c3e940d6ffe9bd21c7397dd583c68bf63 WatchSource:0}: Error finding container d870e3123b77d64f8d05690ba6537b4c3e940d6ffe9bd21c7397dd583c68bf63: Status 404 returned error can't find the container with id d870e3123b77d64f8d05690ba6537b4c3e940d6ffe9bd21c7397dd583c68bf63
Sep 10 21:11:00.363191 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.363147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g" event={"ID":"09a98e1a-65c7-4598-aedb-685f1b4519df","Type":"ContainerStarted","Data":"ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab"}
Sep 10 21:11:00.363644 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.363219    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g" event={"ID":"09a98e1a-65c7-4598-aedb-685f1b4519df","Type":"ContainerStarted","Data":"d870e3123b77d64f8d05690ba6537b4c3e940d6ffe9bd21c7397dd583c68bf63"}
Sep 10 21:11:00.363644 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.363366    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:11:00.364817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.364793    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7" event={"ID":"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf","Type":"ContainerStarted","Data":"97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e"}
Sep 10 21:11:00.364958 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.364830    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7" event={"ID":"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf","Type":"ContainerStarted","Data":"0878ab7dcb4c368dc144ab2d60fe751ad52645cdcf00120d1daa1f47dd7701b1"}
Sep 10 21:11:00.364958 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.364882    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:11:00.366395 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.366374    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs" event={"ID":"1bc215ab-e08c-46f7-a64c-7bf237683b44","Type":"ContainerStarted","Data":"6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9"}
Sep 10 21:11:00.366476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.366417    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs" event={"ID":"1bc215ab-e08c-46f7-a64c-7bf237683b44","Type":"ContainerStarted","Data":"c875cfb315f48000b0d0b9c713fc1961ff0674215aa0393d686d0183afa829ad"}
Sep 10 21:11:00.366476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.366459    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:11:00.382615 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.382570    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g" podStartSLOduration=1.382551215 podStartE2EDuration="1.382551215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:00.382055 +0000 UTC m=+740.401460354" watchObservedRunningTime="2026-09-10 21:11:00.382551215 +0000 UTC m=+740.401956538"
Sep 10 21:11:00.404742 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.404675    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7" podStartSLOduration=1.404656984 podStartE2EDuration="1.404656984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:00.402702786 +0000 UTC m=+740.422108125" watchObservedRunningTime="2026-09-10 21:11:00.404656984 +0000 UTC m=+740.424062305"
Sep 10 21:11:00.423655 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:00.423601    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs" podStartSLOduration=2.032642137 podStartE2EDuration="2.42358574s" totalImagesPullingTime="390.943603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:00.42357661 +0000 UTC m=+740.442981929" watchObservedRunningTime="2026-09-10 21:11:00.42358574 +0000 UTC m=+740.442991080"
Sep 10 21:11:06.379180 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:06.379139    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:11:09.668310 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:09.668271    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:11:11.375482 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:11.375447    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:11:11.376037 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:11.375698    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:11:27.751605 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:27.751571    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:11:29.937876 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:29.937834    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"]
Sep 10 21:11:29.938349 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:29.938062    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs" podUID="1bc215ab-e08c-46f7-a64c-7bf237683b44" containerName="maas-api" containerID="cri-o://6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9" gracePeriod=30
Sep 10 21:11:30.012618 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.012585    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"]
Sep 10 21:11:30.012857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.012832    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g" podUID="09a98e1a-65c7-4598-aedb-685f1b4519df" containerName="payload-processing" containerID="cri-o://ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab" gracePeriod=30
Sep 10 21:11:30.027014 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.026985    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"]
Sep 10 21:11:30.027264 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.027232    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7" podUID="9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" containerName="payload-pre-processing" containerID="cri-o://97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e" gracePeriod=30
Sep 10 21:11:30.292328 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.292301    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:11:30.294168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.294146    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:11:30.326830 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.326803    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:11:30.345379 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.345359    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-metrics-certs\") pod \"1bc215ab-e08c-46f7-a64c-7bf237683b44\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") "
Sep 10 21:11:30.345492 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.345401    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shpsv\" (UniqueName: \"kubernetes.io/projected/1bc215ab-e08c-46f7-a64c-7bf237683b44-kube-api-access-shpsv\") pod \"1bc215ab-e08c-46f7-a64c-7bf237683b44\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") "
Sep 10 21:11:30.345492 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.345420    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls\") pod \"1bc215ab-e08c-46f7-a64c-7bf237683b44\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") "
Sep 10 21:11:30.345492 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.345463    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kc6cv\" (UniqueName: \"kubernetes.io/projected/09a98e1a-65c7-4598-aedb-685f1b4519df-kube-api-access-kc6cv\") pod \"09a98e1a-65c7-4598-aedb-685f1b4519df\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") "
Sep 10 21:11:30.345492 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.345488    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a98e1a-65c7-4598-aedb-685f1b4519df-plugins-config-volume\") pod \"09a98e1a-65c7-4598-aedb-685f1b4519df\" (UID: \"09a98e1a-65c7-4598-aedb-685f1b4519df\") "
Sep 10 21:11:30.346193 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.345520    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc215ab-e08c-46f7-a64c-7bf237683b44-openshift-service-ca\") pod \"1bc215ab-e08c-46f7-a64c-7bf237683b44\" (UID: \"1bc215ab-e08c-46f7-a64c-7bf237683b44\") "
Sep 10 21:11:30.346193 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.345974    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a98e1a-65c7-4598-aedb-685f1b4519df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "09a98e1a-65c7-4598-aedb-685f1b4519df" (UID: "09a98e1a-65c7-4598-aedb-685f1b4519df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:11:30.346193 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.346088    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bc215ab-e08c-46f7-a64c-7bf237683b44-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1bc215ab-e08c-46f7-a64c-7bf237683b44" (UID: "1bc215ab-e08c-46f7-a64c-7bf237683b44"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:11:30.348863 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.348386    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1bc215ab-e08c-46f7-a64c-7bf237683b44" (UID: "1bc215ab-e08c-46f7-a64c-7bf237683b44"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:11:30.348863 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.348644    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bc215ab-e08c-46f7-a64c-7bf237683b44-kube-api-access-shpsv" (OuterVolumeSpecName: "kube-api-access-shpsv") pod "1bc215ab-e08c-46f7-a64c-7bf237683b44" (UID: "1bc215ab-e08c-46f7-a64c-7bf237683b44"). InnerVolumeSpecName "kube-api-access-shpsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:11:30.350002 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.349855    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a98e1a-65c7-4598-aedb-685f1b4519df-kube-api-access-kc6cv" (OuterVolumeSpecName: "kube-api-access-kc6cv") pod "09a98e1a-65c7-4598-aedb-685f1b4519df" (UID: "09a98e1a-65c7-4598-aedb-685f1b4519df"). InnerVolumeSpecName "kube-api-access-kc6cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:11:30.350002 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.349964    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1bc215ab-e08c-46f7-a64c-7bf237683b44" (UID: "1bc215ab-e08c-46f7-a64c-7bf237683b44"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:11:30.447124 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447051    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-plugins-config-volume\") pod \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") "
Sep 10 21:11:30.447124 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447122    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bhwr\" (UniqueName: \"kubernetes.io/projected/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-kube-api-access-4bhwr\") pod \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\" (UID: \"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf\") "
Sep 10 21:11:30.447305 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447294    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.447340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447304    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shpsv\" (UniqueName: \"kubernetes.io/projected/1bc215ab-e08c-46f7-a64c-7bf237683b44-kube-api-access-shpsv\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.447340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447314    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1bc215ab-e08c-46f7-a64c-7bf237683b44-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.447340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447323    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kc6cv\" (UniqueName: \"kubernetes.io/projected/09a98e1a-65c7-4598-aedb-685f1b4519df-kube-api-access-kc6cv\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.447340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447332    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09a98e1a-65c7-4598-aedb-685f1b4519df-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.447466 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447340    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1bc215ab-e08c-46f7-a64c-7bf237683b44-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.447466 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.447414    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" (UID: "9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:11:30.449001 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.448981    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-kube-api-access-4bhwr" (OuterVolumeSpecName: "kube-api-access-4bhwr") pod "9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" (UID: "9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf"). InnerVolumeSpecName "kube-api-access-4bhwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:11:30.537384 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.537356    2557 generic.go:356] "Generic (PLEG): container finished" podID="09a98e1a-65c7-4598-aedb-685f1b4519df" containerID="ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab" exitCode=0
Sep 10 21:11:30.537563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.537401    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"
Sep 10 21:11:30.537563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.537442    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g" event={"ID":"09a98e1a-65c7-4598-aedb-685f1b4519df","Type":"ContainerDied","Data":"ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab"}
Sep 10 21:11:30.537563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.537488    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g" event={"ID":"09a98e1a-65c7-4598-aedb-685f1b4519df","Type":"ContainerDied","Data":"d870e3123b77d64f8d05690ba6537b4c3e940d6ffe9bd21c7397dd583c68bf63"}
Sep 10 21:11:30.537563 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.537513    2557 scope.go:117] "RemoveContainer" containerID="ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab"
Sep 10 21:11:30.538895 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.538877    2557 generic.go:356] "Generic (PLEG): container finished" podID="9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" containerID="97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e" exitCode=0
Sep 10 21:11:30.539012 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.538921    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"
Sep 10 21:11:30.539012 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.538963    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7" event={"ID":"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf","Type":"ContainerDied","Data":"97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e"}
Sep 10 21:11:30.539012 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.539007    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7" event={"ID":"9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf","Type":"ContainerDied","Data":"0878ab7dcb4c368dc144ab2d60fe751ad52645cdcf00120d1daa1f47dd7701b1"}
Sep 10 21:11:30.540530 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.540495    2557 generic.go:356] "Generic (PLEG): container finished" podID="1bc215ab-e08c-46f7-a64c-7bf237683b44" containerID="6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9" exitCode=0
Sep 10 21:11:30.540628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.540523    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs" event={"ID":"1bc215ab-e08c-46f7-a64c-7bf237683b44","Type":"ContainerDied","Data":"6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9"}
Sep 10 21:11:30.540628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.540560    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"
Sep 10 21:11:30.540628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.540576    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs" event={"ID":"1bc215ab-e08c-46f7-a64c-7bf237683b44","Type":"ContainerDied","Data":"c875cfb315f48000b0d0b9c713fc1961ff0674215aa0393d686d0183afa829ad"}
Sep 10 21:11:30.547817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.547791    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bhwr\" (UniqueName: \"kubernetes.io/projected/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-kube-api-access-4bhwr\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.547910 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.547822    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:11:30.549420 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.549403    2557 scope.go:117] "RemoveContainer" containerID="ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab"
Sep 10 21:11:30.549754 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:11:30.549725    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab\": container with ID starting with ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab not found: ID does not exist" containerID="ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab"
Sep 10 21:11:30.549841 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.549764    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab"} err="failed to get container status \"ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab\": rpc error: code = NotFound desc = could not find container \"ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab\": container with ID starting with ef31e79cdebf2a238241d3b99037b92c25fb6fe9dbec544a2aead565097737ab not found: ID does not exist"
Sep 10 21:11:30.549841 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.549785    2557 scope.go:117] "RemoveContainer" containerID="97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e"
Sep 10 21:11:30.560109 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.560091    2557 scope.go:117] "RemoveContainer" containerID="97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e"
Sep 10 21:11:30.560531 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:11:30.560509    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e\": container with ID starting with 97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e not found: ID does not exist" containerID="97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e"
Sep 10 21:11:30.560615 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.560536    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e"} err="failed to get container status \"97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e\": rpc error: code = NotFound desc = could not find container \"97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e\": container with ID starting with 97412fbf90d96816b5522f5f69f37e8196ae987dc8b76ff8318b784cb52c771e not found: ID does not exist"
Sep 10 21:11:30.560615 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.560552    2557 scope.go:117] "RemoveContainer" containerID="6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9"
Sep 10 21:11:30.569217 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.569204    2557 scope.go:117] "RemoveContainer" containerID="6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9"
Sep 10 21:11:30.569440 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:11:30.569421    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9\": container with ID starting with 6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9 not found: ID does not exist" containerID="6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9"
Sep 10 21:11:30.569511 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.569446    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9"} err="failed to get container status \"6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9\": rpc error: code = NotFound desc = could not find container \"6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9\": container with ID starting with 6228650681faed7b179ff066a5e471c081602de1f1d583b77ed40e1da2108fe9 not found: ID does not exist"
Sep 10 21:11:30.591039 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.591014    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"]
Sep 10 21:11:30.593340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.593320    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-bbddbd71-5947799886-2skb7"]
Sep 10 21:11:30.610924 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.610900    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"]
Sep 10 21:11:30.617707 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.617671    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-bbddbd71-6cd5b4f84b-pfd4g"]
Sep 10 21:11:30.634313 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.634291    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"]
Sep 10 21:11:30.648858 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:30.648825    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-bbddbd71-778d449b45-fjcbs"]
Sep 10 21:11:32.585576 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:32.585546    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09a98e1a-65c7-4598-aedb-685f1b4519df" path="/var/lib/kubelet/pods/09a98e1a-65c7-4598-aedb-685f1b4519df/volumes"
Sep 10 21:11:32.585957 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:32.585936    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bc215ab-e08c-46f7-a64c-7bf237683b44" path="/var/lib/kubelet/pods/1bc215ab-e08c-46f7-a64c-7bf237683b44/volumes"
Sep 10 21:11:32.586248 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:32.586235    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" path="/var/lib/kubelet/pods/9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf/volumes"
Sep 10 21:11:35.386558 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.386524    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"]
Sep 10 21:11:35.387034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.386941    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a98e1a-65c7-4598-aedb-685f1b4519df" containerName="payload-processing"
Sep 10 21:11:35.387034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.386955    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a98e1a-65c7-4598-aedb-685f1b4519df" containerName="payload-processing"
Sep 10 21:11:35.387034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.386983    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bc215ab-e08c-46f7-a64c-7bf237683b44" containerName="maas-api"
Sep 10 21:11:35.387034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.386990    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bc215ab-e08c-46f7-a64c-7bf237683b44" containerName="maas-api"
Sep 10 21:11:35.387034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.386999    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" containerName="payload-pre-processing"
Sep 10 21:11:35.387034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.387005    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" containerName="payload-pre-processing"
Sep 10 21:11:35.387338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.387070    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bc215ab-e08c-46f7-a64c-7bf237683b44" containerName="maas-api"
Sep 10 21:11:35.387338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.387080    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a98e1a-65c7-4598-aedb-685f1b4519df" containerName="payload-processing"
Sep 10 21:11:35.387338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.387088    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ca8dcc6-817b-4f6b-bed5-e89cd07ab8cf" containerName="payload-pre-processing"
Sep 10 21:11:35.390909 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.390888    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.398803 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.398779    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:11:35.398935 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.398897    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:11:35.399055 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.399021    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e95f46e9-serving-cert\""
Sep 10 21:11:35.399055 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.399030    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:11:35.399923 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.399905    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:11:35.423572 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.423548    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"]
Sep 10 21:11:35.491558 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.491526    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmddt\" (UniqueName: \"kubernetes.io/projected/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-kube-api-access-qmddt\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.491751 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.491570    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-openshift-service-ca\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.491751 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.491604    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-metrics-certs\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.491751 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.491624    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.592432 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.592397    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-openshift-service-ca\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.592432 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.592435    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-metrics-certs\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.592662 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.592452    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.592662 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.592503    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmddt\" (UniqueName: \"kubernetes.io/projected/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-kube-api-access-qmddt\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.592662 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:11:35.592610    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-serving-cert: secret "maas-api-e2e-ait-e95f46e9-serving-cert" not found
Sep 10 21:11:35.592862 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:11:35.592677    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls podName:10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9 nodeName:}" failed. No retries permitted until 2026-09-10 21:11:36.092662043 +0000 UTC m=+776.112067344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls") pod "maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv" (UID: "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9") : secret "maas-api-e2e-ait-e95f46e9-serving-cert" not found
Sep 10 21:11:35.593170 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.593151    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-openshift-service-ca\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.594819 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.594801    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-metrics-certs\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:35.624500 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:35.624477    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmddt\" (UniqueName: \"kubernetes.io/projected/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-kube-api-access-qmddt\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:36.097016 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.096987    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:36.099368 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.099344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls\") pod \"maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:36.301507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.301471    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:36.422797 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.422767    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"]
Sep 10 21:11:36.427423 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.427390    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:36.432858 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.432834    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-e95f46e9\""
Sep 10 21:11:36.433009 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.432914    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-e95f46e9-dockercfg-9t289\""
Sep 10 21:11:36.441508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.441486    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"]
Sep 10 21:11:36.500244 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.500212    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-plugins-config-volume\") pod \"payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") " pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:36.500244 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.500245    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5htsp\" (UniqueName: \"kubernetes.io/projected/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-kube-api-access-5htsp\") pod \"payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") " pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:36.578965 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.578925    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"]
Sep 10 21:11:36.580436 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:11:36.580410    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10f08f6d_59b5_41b8_83dc_cbd2aa8e3af9.slice/crio-0edc100c3b1483b48c959c3d868d26b53fe3ef0bcad4c59934fc34d13ff5861f WatchSource:0}: Error finding container 0edc100c3b1483b48c959c3d868d26b53fe3ef0bcad4c59934fc34d13ff5861f: Status 404 returned error can't find the container with id 0edc100c3b1483b48c959c3d868d26b53fe3ef0bcad4c59934fc34d13ff5861f
Sep 10 21:11:36.601727 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.601652    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-plugins-config-volume\") pod \"payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") " pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:36.601727 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.601714    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5htsp\" (UniqueName: \"kubernetes.io/projected/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-kube-api-access-5htsp\") pod \"payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") " pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:36.602428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.602407    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-plugins-config-volume\") pod \"payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") " pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:36.612643 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.612618    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5htsp\" (UniqueName: \"kubernetes.io/projected/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-kube-api-access-5htsp\") pod \"payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") " pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:36.740387 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:36.740364    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:37.015007 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.014982    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"]
Sep 10 21:11:37.015612 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:11:37.015590    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ca91c2_5a95_4546_b066_ba52bdbd2d2c.slice/crio-403b5ca69dba64ddf6db658e7e47926df91f5eca9c30b387731438baf679d1b4 WatchSource:0}: Error finding container 403b5ca69dba64ddf6db658e7e47926df91f5eca9c30b387731438baf679d1b4: Status 404 returned error can't find the container with id 403b5ca69dba64ddf6db658e7e47926df91f5eca9c30b387731438baf679d1b4
Sep 10 21:11:37.585408 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.585374    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv" event={"ID":"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9","Type":"ContainerStarted","Data":"3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79"}
Sep 10 21:11:37.585869 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.585429    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv" event={"ID":"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9","Type":"ContainerStarted","Data":"0edc100c3b1483b48c959c3d868d26b53fe3ef0bcad4c59934fc34d13ff5861f"}
Sep 10 21:11:37.585869 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.585472    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:37.587153 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.587128    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8" event={"ID":"41ca91c2-5a95-4546-b066-ba52bdbd2d2c","Type":"ContainerStarted","Data":"6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b"}
Sep 10 21:11:37.587307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.587163    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8" event={"ID":"41ca91c2-5a95-4546-b066-ba52bdbd2d2c","Type":"ContainerStarted","Data":"403b5ca69dba64ddf6db658e7e47926df91f5eca9c30b387731438baf679d1b4"}
Sep 10 21:11:37.587307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.587198    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:37.617575 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.617534    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv" podStartSLOduration=2.117439027 podStartE2EDuration="2.617521932s" totalImagesPullingTime="500.082905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:37.616568735 +0000 UTC m=+777.635974055" watchObservedRunningTime="2026-09-10 21:11:37.617521932 +0000 UTC m=+777.636927284"
Sep 10 21:11:37.651384 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:37.651340    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8" podStartSLOduration=1.6513273370000001 podStartE2EDuration="1.651327337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:11:37.650760172 +0000 UTC m=+777.670165494" watchObservedRunningTime="2026-09-10 21:11:37.651327337 +0000 UTC m=+777.670732655"
Sep 10 21:11:42.140776 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:42.140731    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:11:43.598659 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:43.598628    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:11:48.595105 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:48.595076    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:11:57.343814 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:11:57.343773    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:12:06.174822 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.174741    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"]
Sep 10 21:12:06.175599 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.175001    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv" podUID="10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" containerName="maas-api" containerID="cri-o://3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79" gracePeriod=30
Sep 10 21:12:06.221625 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.221594    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"]
Sep 10 21:12:06.221911 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.221844    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8" podUID="41ca91c2-5a95-4546-b066-ba52bdbd2d2c" containerName="payload-processing" containerID="cri-o://6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b" gracePeriod=30
Sep 10 21:12:06.463089 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.463065    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:12:06.488874 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.488847    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:12:06.544330 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544303    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-openshift-service-ca\") pod \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") "
Sep 10 21:12:06.544330 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544333    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls\") pod \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") "
Sep 10 21:12:06.544577 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544352    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-plugins-config-volume\") pod \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") "
Sep 10 21:12:06.544577 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544372    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-metrics-certs\") pod \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") "
Sep 10 21:12:06.544577 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544401    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmddt\" (UniqueName: \"kubernetes.io/projected/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-kube-api-access-qmddt\") pod \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\" (UID: \"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9\") "
Sep 10 21:12:06.544577 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544485    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5htsp\" (UniqueName: \"kubernetes.io/projected/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-kube-api-access-5htsp\") pod \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\" (UID: \"41ca91c2-5a95-4546-b066-ba52bdbd2d2c\") "
Sep 10 21:12:06.544863 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544765    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" (UID: "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:06.544863 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544782    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "41ca91c2-5a95-4546-b066-ba52bdbd2d2c" (UID: "41ca91c2-5a95-4546-b066-ba52bdbd2d2c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:06.544978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544902    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:06.544978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.544923    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:06.546498 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.546475    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-kube-api-access-qmddt" (OuterVolumeSpecName: "kube-api-access-qmddt") pod "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" (UID: "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9"). InnerVolumeSpecName "kube-api-access-qmddt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:06.546653 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.546632    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-kube-api-access-5htsp" (OuterVolumeSpecName: "kube-api-access-5htsp") pod "41ca91c2-5a95-4546-b066-ba52bdbd2d2c" (UID: "41ca91c2-5a95-4546-b066-ba52bdbd2d2c"). InnerVolumeSpecName "kube-api-access-5htsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:06.546941 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.546919    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" (UID: "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:06.547020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.546942    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" (UID: "10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:06.646203 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.646175    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:06.646203 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.646197    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:06.646203 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.646206    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmddt\" (UniqueName: \"kubernetes.io/projected/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9-kube-api-access-qmddt\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:06.646460 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.646215    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5htsp\" (UniqueName: \"kubernetes.io/projected/41ca91c2-5a95-4546-b066-ba52bdbd2d2c-kube-api-access-5htsp\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:06.742243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.742211    2557 generic.go:356] "Generic (PLEG): container finished" podID="10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" containerID="3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79" exitCode=0
Sep 10 21:12:06.742396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.742248    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"
Sep 10 21:12:06.742396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.742249    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv" event={"ID":"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9","Type":"ContainerDied","Data":"3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79"}
Sep 10 21:12:06.742396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.742305    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv" event={"ID":"10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9","Type":"ContainerDied","Data":"0edc100c3b1483b48c959c3d868d26b53fe3ef0bcad4c59934fc34d13ff5861f"}
Sep 10 21:12:06.742396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.742330    2557 scope.go:117] "RemoveContainer" containerID="3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79"
Sep 10 21:12:06.743809 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.743782    2557 generic.go:356] "Generic (PLEG): container finished" podID="41ca91c2-5a95-4546-b066-ba52bdbd2d2c" containerID="6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b" exitCode=0
Sep 10 21:12:06.743959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.743827    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"
Sep 10 21:12:06.743959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.743911    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8" event={"ID":"41ca91c2-5a95-4546-b066-ba52bdbd2d2c","Type":"ContainerDied","Data":"6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b"}
Sep 10 21:12:06.743959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.743956    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8" event={"ID":"41ca91c2-5a95-4546-b066-ba52bdbd2d2c","Type":"ContainerDied","Data":"403b5ca69dba64ddf6db658e7e47926df91f5eca9c30b387731438baf679d1b4"}
Sep 10 21:12:06.751403 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.751383    2557 scope.go:117] "RemoveContainer" containerID="3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79"
Sep 10 21:12:06.751664 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:06.751647    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79\": container with ID starting with 3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79 not found: ID does not exist" containerID="3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79"
Sep 10 21:12:06.751737 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.751670    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79"} err="failed to get container status \"3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79\": rpc error: code = NotFound desc = could not find container \"3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79\": container with ID starting with 3fd244ca85761f18f696e541ed58365c1fa28892fdaa39071bb6416d83180e79 not found: ID does not exist"
Sep 10 21:12:06.751737 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.751698    2557 scope.go:117] "RemoveContainer" containerID="6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b"
Sep 10 21:12:06.760338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.760317    2557 scope.go:117] "RemoveContainer" containerID="6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b"
Sep 10 21:12:06.760623 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:06.760583    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b\": container with ID starting with 6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b not found: ID does not exist" containerID="6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b"
Sep 10 21:12:06.760705 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.760630    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b"} err="failed to get container status \"6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b\": rpc error: code = NotFound desc = could not find container \"6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b\": container with ID starting with 6b0d5fff70e633a9f7ad760a53fcdd84c6104e9d967b15da82d3ae2b8858cb2b not found: ID does not exist"
Sep 10 21:12:06.760814 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.760802    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"]
Sep 10 21:12:06.763037 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.763016    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e95f46e9-6465ccc58f-hg4fv"]
Sep 10 21:12:06.773645 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.773622    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"]
Sep 10 21:12:06.777133 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:06.777113    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-e95f46e9-687b6bf445-d6vp8"]
Sep 10 21:12:08.586063 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:08.586035    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" path="/var/lib/kubelet/pods/10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9/volumes"
Sep 10 21:12:08.586414 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:08.586366    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ca91c2-5a95-4546-b066-ba52bdbd2d2c" path="/var/lib/kubelet/pods/41ca91c2-5a95-4546-b066-ba52bdbd2d2c/volumes"
Sep 10 21:12:12.547693 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.547639    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"]
Sep 10 21:12:12.548235 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.548217    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ca91c2-5a95-4546-b066-ba52bdbd2d2c" containerName="payload-processing"
Sep 10 21:12:12.548307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.548238    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ca91c2-5a95-4546-b066-ba52bdbd2d2c" containerName="payload-processing"
Sep 10 21:12:12.548307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.548278    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" containerName="maas-api"
Sep 10 21:12:12.548307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.548287    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" containerName="maas-api"
Sep 10 21:12:12.548462 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.548380    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="10f08f6d-59b5-41b8-83dc-cbd2aa8e3af9" containerName="maas-api"
Sep 10 21:12:12.548462 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.548395    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41ca91c2-5a95-4546-b066-ba52bdbd2d2c" containerName="payload-processing"
Sep 10 21:12:12.553104 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.553081    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.555571 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.555552    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:12:12.558288 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.558264    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-0d892491-serving-cert\""
Sep 10 21:12:12.558397 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.558341    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:12:12.559404 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.559303    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:12:12.559524 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.559404    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:12:12.567785 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.567761    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"]
Sep 10 21:12:12.594968 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.594947    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlrh8\" (UniqueName: \"kubernetes.io/projected/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-kube-api-access-dlrh8\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.595152 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.594990    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-openshift-service-ca\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.595152 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.595115    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-metrics-certs\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.595275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.595154    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.696412 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.696384    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-metrics-certs\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.696519 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.696426    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.696519 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.696489    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlrh8\" (UniqueName: \"kubernetes.io/projected/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-kube-api-access-dlrh8\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.696604 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.696543    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-openshift-service-ca\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.696604 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:12.696578    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-serving-cert: secret "maas-api-e2e-derive-0d892491-serving-cert" not found
Sep 10 21:12:12.696717 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:12.696633    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls podName:9d5eb3dd-a73f-4da4-8c97-403b1304dad8 nodeName:}" failed. No retries permitted until 2026-09-10 21:12:13.19661771 +0000 UTC m=+813.216023011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls") pod "maas-api-e2e-derive-0d892491-d8599d944-9pnzm" (UID: "9d5eb3dd-a73f-4da4-8c97-403b1304dad8") : secret "maas-api-e2e-derive-0d892491-serving-cert" not found
Sep 10 21:12:12.697329 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.697304    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-openshift-service-ca\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.698747 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.698724    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-metrics-certs\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:12.718850 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:12.718830    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlrh8\" (UniqueName: \"kubernetes.io/projected/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-kube-api-access-dlrh8\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:13.200588 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.200553    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:13.202962 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.202930    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls\") pod \"maas-api-e2e-derive-0d892491-d8599d944-9pnzm\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:13.253910 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.253879    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:12:13.319354 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.319324    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"]
Sep 10 21:12:13.323113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.323097    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.329419 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.329398    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-0d892491-dockercfg-xk5wj\""
Sep 10 21:12:13.332188 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.332169    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-0d892491\""
Sep 10 21:12:13.336163 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.336144    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"]
Sep 10 21:12:13.369362 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.369335    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"]
Sep 10 21:12:13.372791 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.372776    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.383244 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.383221    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"]
Sep 10 21:12:13.402759 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.402735    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cnb\" (UniqueName: \"kubernetes.io/projected/cec4c795-89fc-4e9d-934f-b80f81a8db7c-kube-api-access-t4cnb\") pod \"payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.402759 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.402764    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-plugins-config-volume\") pod \"payload-processing-e2e-derive-0d892491-7c4995f89-69t76\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") " pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.402908 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.402796    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cec4c795-89fc-4e9d-934f-b80f81a8db7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.402947 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.402917    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbl8f\" (UniqueName: \"kubernetes.io/projected/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-kube-api-access-mbl8f\") pod \"payload-processing-e2e-derive-0d892491-7c4995f89-69t76\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") " pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.464795 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.464764    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:13.503837 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.503807    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cnb\" (UniqueName: \"kubernetes.io/projected/cec4c795-89fc-4e9d-934f-b80f81a8db7c-kube-api-access-t4cnb\") pod \"payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.503941 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.503846    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-plugins-config-volume\") pod \"payload-processing-e2e-derive-0d892491-7c4995f89-69t76\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") " pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.504034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.504014    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cec4c795-89fc-4e9d-934f-b80f81a8db7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.504103 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.504089    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbl8f\" (UniqueName: \"kubernetes.io/projected/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-kube-api-access-mbl8f\") pod \"payload-processing-e2e-derive-0d892491-7c4995f89-69t76\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") " pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.504710 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.504673    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-plugins-config-volume\") pod \"payload-processing-e2e-derive-0d892491-7c4995f89-69t76\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") " pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.504810 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.504741    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cec4c795-89fc-4e9d-934f-b80f81a8db7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.513446 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.513372    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cnb\" (UniqueName: \"kubernetes.io/projected/cec4c795-89fc-4e9d-934f-b80f81a8db7c-kube-api-access-t4cnb\") pod \"payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.513863 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.513791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbl8f\" (UniqueName: \"kubernetes.io/projected/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-kube-api-access-mbl8f\") pod \"payload-processing-e2e-derive-0d892491-7c4995f89-69t76\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") " pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.634717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.634667    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:13.683086 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.682980    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:13.739526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.739403    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"]
Sep 10 21:12:13.785315 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.785280    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm" event={"ID":"9d5eb3dd-a73f-4da4-8c97-403b1304dad8","Type":"ContainerStarted","Data":"5f335c4a1f5052e1aa48f5f5cd03e29da2711f8be388195f9ef7f93b5ae5770e"}
Sep 10 21:12:13.908637 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.908610    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"]
Sep 10 21:12:13.909527 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:12:13.909501    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab8aeb7c_00c8_467a_955f_4bf5da0ebf8d.slice/crio-d4dd755d9a389539b2997c22a21831c40ba2a8e0c339ebec09bb07d0460da06f WatchSource:0}: Error finding container d4dd755d9a389539b2997c22a21831c40ba2a8e0c339ebec09bb07d0460da06f: Status 404 returned error can't find the container with id d4dd755d9a389539b2997c22a21831c40ba2a8e0c339ebec09bb07d0460da06f
Sep 10 21:12:13.962484 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:13.962461    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"]
Sep 10 21:12:13.963527 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:12:13.963501    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec4c795_89fc_4e9d_934f_b80f81a8db7c.slice/crio-813cfdb94b31c9e39671c809c8aec3571966c4f169794e058c5c06c615f997c5 WatchSource:0}: Error finding container 813cfdb94b31c9e39671c809c8aec3571966c4f169794e058c5c06c615f997c5: Status 404 returned error can't find the container with id 813cfdb94b31c9e39671c809c8aec3571966c4f169794e058c5c06c615f997c5
Sep 10 21:12:14.793608 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.793555    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76" event={"ID":"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d","Type":"ContainerStarted","Data":"0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9"}
Sep 10 21:12:14.794090 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.793626    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76" event={"ID":"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d","Type":"ContainerStarted","Data":"d4dd755d9a389539b2997c22a21831c40ba2a8e0c339ebec09bb07d0460da06f"}
Sep 10 21:12:14.794090 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.793656    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:14.795332 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.795302    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5" event={"ID":"cec4c795-89fc-4e9d-934f-b80f81a8db7c","Type":"ContainerStarted","Data":"1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968"}
Sep 10 21:12:14.795451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.795344    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5" event={"ID":"cec4c795-89fc-4e9d-934f-b80f81a8db7c","Type":"ContainerStarted","Data":"813cfdb94b31c9e39671c809c8aec3571966c4f169794e058c5c06c615f997c5"}
Sep 10 21:12:14.795451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.795370    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:14.797027 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.797004    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm" event={"ID":"9d5eb3dd-a73f-4da4-8c97-403b1304dad8","Type":"ContainerStarted","Data":"99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0"}
Sep 10 21:12:14.797153 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.797123    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:14.814308 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.814259    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76" podStartSLOduration=1.814245903 podStartE2EDuration="1.814245903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:14.81134545 +0000 UTC m=+814.830750770" watchObservedRunningTime="2026-09-10 21:12:14.814245903 +0000 UTC m=+814.833651222"
Sep 10 21:12:14.831487 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.831449    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm" podStartSLOduration=2.364596288 podStartE2EDuration="2.831434238s" totalImagesPullingTime="466.83795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:14.830381356 +0000 UTC m=+814.849786678" watchObservedRunningTime="2026-09-10 21:12:14.831434238 +0000 UTC m=+814.850839567"
Sep 10 21:12:14.851674 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:14.851635    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5" podStartSLOduration=1.851624251 podStartE2EDuration="1.851624251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:14.851055869 +0000 UTC m=+814.870461188" watchObservedRunningTime="2026-09-10 21:12:14.851624251 +0000 UTC m=+814.871029569"
Sep 10 21:12:20.807803 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:20.807774    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:25.552212 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:25.552178    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:12:25.804829 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:25.804740    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:25.805119 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:25.805098    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:42.289982 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.289937    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"]
Sep 10 21:12:42.290519 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.290311    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4" podUID="72634e55-dc83-4bc7-9185-d3fc4f50bb9c" containerName="payload-processing" containerID="cri-o://b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649" gracePeriod=30
Sep 10 21:12:42.296294 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.296267    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"]
Sep 10 21:12:42.296523 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.296499    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98" podUID="c9d4c5e5-467f-4678-802d-5d500e1ec283" containerName="payload-pre-processing" containerID="cri-o://8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38" gracePeriod=30
Sep 10 21:12:42.544485 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.544430    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:12:42.578379 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.578354    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:12:42.664499 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.664473    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2c8h\" (UniqueName: \"kubernetes.io/projected/c9d4c5e5-467f-4678-802d-5d500e1ec283-kube-api-access-c2c8h\") pod \"c9d4c5e5-467f-4678-802d-5d500e1ec283\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") "
Sep 10 21:12:42.664499 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.664501    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59865\" (UniqueName: \"kubernetes.io/projected/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-kube-api-access-59865\") pod \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") "
Sep 10 21:12:42.664756 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.664527    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d4c5e5-467f-4678-802d-5d500e1ec283-plugins-config-volume\") pod \"c9d4c5e5-467f-4678-802d-5d500e1ec283\" (UID: \"c9d4c5e5-467f-4678-802d-5d500e1ec283\") "
Sep 10 21:12:42.664756 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.664548    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-plugins-config-volume\") pod \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\" (UID: \"72634e55-dc83-4bc7-9185-d3fc4f50bb9c\") "
Sep 10 21:12:42.664948 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.664922    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9d4c5e5-467f-4678-802d-5d500e1ec283-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c9d4c5e5-467f-4678-802d-5d500e1ec283" (UID: "c9d4c5e5-467f-4678-802d-5d500e1ec283"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:42.665021 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.664983    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72634e55-dc83-4bc7-9185-d3fc4f50bb9c" (UID: "72634e55-dc83-4bc7-9185-d3fc4f50bb9c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:42.666591 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.666566    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-kube-api-access-59865" (OuterVolumeSpecName: "kube-api-access-59865") pod "72634e55-dc83-4bc7-9185-d3fc4f50bb9c" (UID: "72634e55-dc83-4bc7-9185-d3fc4f50bb9c"). InnerVolumeSpecName "kube-api-access-59865". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:42.666699 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.666597    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9d4c5e5-467f-4678-802d-5d500e1ec283-kube-api-access-c2c8h" (OuterVolumeSpecName: "kube-api-access-c2c8h") pod "c9d4c5e5-467f-4678-802d-5d500e1ec283" (UID: "c9d4c5e5-467f-4678-802d-5d500e1ec283"). InnerVolumeSpecName "kube-api-access-c2c8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:42.765633 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.765609    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c9d4c5e5-467f-4678-802d-5d500e1ec283-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.765633 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.765629    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.765790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.765638    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2c8h\" (UniqueName: \"kubernetes.io/projected/c9d4c5e5-467f-4678-802d-5d500e1ec283-kube-api-access-c2c8h\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.765790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.765647    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59865\" (UniqueName: \"kubernetes.io/projected/72634e55-dc83-4bc7-9185-d3fc4f50bb9c-kube-api-access-59865\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:42.958960 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.958892    2557 generic.go:356] "Generic (PLEG): container finished" podID="72634e55-dc83-4bc7-9185-d3fc4f50bb9c" containerID="b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649" exitCode=0
Sep 10 21:12:42.958960 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.958934    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"
Sep 10 21:12:42.959161 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.958966    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4" event={"ID":"72634e55-dc83-4bc7-9185-d3fc4f50bb9c","Type":"ContainerDied","Data":"b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649"}
Sep 10 21:12:42.959161 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.959022    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4" event={"ID":"72634e55-dc83-4bc7-9185-d3fc4f50bb9c","Type":"ContainerDied","Data":"8e65a74f97b379fcc6ad978ff450f1bb4403df22cb49114a9478fe9610293052"}
Sep 10 21:12:42.959161 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.959086    2557 scope.go:117] "RemoveContainer" containerID="b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649"
Sep 10 21:12:42.960335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.960278    2557 generic.go:356] "Generic (PLEG): container finished" podID="c9d4c5e5-467f-4678-802d-5d500e1ec283" containerID="8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38" exitCode=0
Sep 10 21:12:42.960335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.960310    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98" event={"ID":"c9d4c5e5-467f-4678-802d-5d500e1ec283","Type":"ContainerDied","Data":"8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38"}
Sep 10 21:12:42.960335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.960315    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"
Sep 10 21:12:42.960471 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.960348    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98" event={"ID":"c9d4c5e5-467f-4678-802d-5d500e1ec283","Type":"ContainerDied","Data":"f06f1c896ea9eb346ee6b58e56c474829598ba8e5cd891f235b4630fbaa26e16"}
Sep 10 21:12:42.968933 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.968911    2557 scope.go:117] "RemoveContainer" containerID="b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649"
Sep 10 21:12:42.969208 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:42.969185    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649\": container with ID starting with b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649 not found: ID does not exist" containerID="b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649"
Sep 10 21:12:42.969280 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.969215    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649"} err="failed to get container status \"b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649\": rpc error: code = NotFound desc = could not find container \"b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649\": container with ID starting with b9cb6bf2449c554331793ed07c680eb4ffeb372fd92d275ae532d3859ce31649 not found: ID does not exist"
Sep 10 21:12:42.969280 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.969230    2557 scope.go:117] "RemoveContainer" containerID="8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38"
Sep 10 21:12:42.977130 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.977115    2557 scope.go:117] "RemoveContainer" containerID="8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38"
Sep 10 21:12:42.977384 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:42.977365    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38\": container with ID starting with 8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38 not found: ID does not exist" containerID="8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38"
Sep 10 21:12:42.977448 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.977389    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38"} err="failed to get container status \"8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38\": rpc error: code = NotFound desc = could not find container \"8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38\": container with ID starting with 8e36bf9a000e59c68aa1461e42ec39cbd83ff69a900047bbf2834c7d36c09b38 not found: ID does not exist"
Sep 10 21:12:42.997241 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.997221    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"]
Sep 10 21:12:42.999713 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:42.999695    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f96a7a-747fb78bfd-bcw98"]
Sep 10 21:12:43.013520 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.013497    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"]
Sep 10 21:12:43.017399 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.017379    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f96a7a-884d47dbf-vd7f4"]
Sep 10 21:12:43.322396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.322360    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"]
Sep 10 21:12:43.322833 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.322614    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm" podUID="9d5eb3dd-a73f-4da4-8c97-403b1304dad8" containerName="maas-api" containerID="cri-o://99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0" gracePeriod=30
Sep 10 21:12:43.384736 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.384703    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"]
Sep 10 21:12:43.385069 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.385038    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76" podUID="ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" containerName="payload-processing" containerID="cri-o://0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9" gracePeriod=30
Sep 10 21:12:43.394925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.394455    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"]
Sep 10 21:12:43.394925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.394737    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5" podUID="cec4c795-89fc-4e9d-934f-b80f81a8db7c" containerName="payload-pre-processing" containerID="cri-o://1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968" gracePeriod=30
Sep 10 21:12:43.651855 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.651824    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:43.695705 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.695651    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:43.712042 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.712020    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:43.775016 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.774982    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls\") pod \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") "
Sep 10 21:12:43.775198 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.775106    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlrh8\" (UniqueName: \"kubernetes.io/projected/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-kube-api-access-dlrh8\") pod \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") "
Sep 10 21:12:43.775282 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.775197    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-openshift-service-ca\") pod \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") "
Sep 10 21:12:43.775282 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.775241    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-metrics-certs\") pod \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\" (UID: \"9d5eb3dd-a73f-4da4-8c97-403b1304dad8\") "
Sep 10 21:12:43.775576 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.775551    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9d5eb3dd-a73f-4da4-8c97-403b1304dad8" (UID: "9d5eb3dd-a73f-4da4-8c97-403b1304dad8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:43.777184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.777161    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9d5eb3dd-a73f-4da4-8c97-403b1304dad8" (UID: "9d5eb3dd-a73f-4da4-8c97-403b1304dad8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:43.777184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.777178    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9d5eb3dd-a73f-4da4-8c97-403b1304dad8" (UID: "9d5eb3dd-a73f-4da4-8c97-403b1304dad8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:12:43.777312 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.777164    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-kube-api-access-dlrh8" (OuterVolumeSpecName: "kube-api-access-dlrh8") pod "9d5eb3dd-a73f-4da4-8c97-403b1304dad8" (UID: "9d5eb3dd-a73f-4da4-8c97-403b1304dad8"). InnerVolumeSpecName "kube-api-access-dlrh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:43.876514 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876465    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4cnb\" (UniqueName: \"kubernetes.io/projected/cec4c795-89fc-4e9d-934f-b80f81a8db7c-kube-api-access-t4cnb\") pod \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") "
Sep 10 21:12:43.876514 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876519    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cec4c795-89fc-4e9d-934f-b80f81a8db7c-plugins-config-volume\") pod \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\" (UID: \"cec4c795-89fc-4e9d-934f-b80f81a8db7c\") "
Sep 10 21:12:43.876643 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876545    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-plugins-config-volume\") pod \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") "
Sep 10 21:12:43.876643 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876564    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbl8f\" (UniqueName: \"kubernetes.io/projected/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-kube-api-access-mbl8f\") pod \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\" (UID: \"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d\") "
Sep 10 21:12:43.876773 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876757    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.876817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876779    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.876817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876793    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.876817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876806    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlrh8\" (UniqueName: \"kubernetes.io/projected/9d5eb3dd-a73f-4da4-8c97-403b1304dad8-kube-api-access-dlrh8\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.876957 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876876    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cec4c795-89fc-4e9d-934f-b80f81a8db7c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cec4c795-89fc-4e9d-934f-b80f81a8db7c" (UID: "cec4c795-89fc-4e9d-934f-b80f81a8db7c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:43.876957 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.876891    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" (UID: "ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:12:43.878340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.878323    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cec4c795-89fc-4e9d-934f-b80f81a8db7c-kube-api-access-t4cnb" (OuterVolumeSpecName: "kube-api-access-t4cnb") pod "cec4c795-89fc-4e9d-934f-b80f81a8db7c" (UID: "cec4c795-89fc-4e9d-934f-b80f81a8db7c"). InnerVolumeSpecName "kube-api-access-t4cnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:43.878388 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.878372    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-kube-api-access-mbl8f" (OuterVolumeSpecName: "kube-api-access-mbl8f") pod "ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" (UID: "ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d"). InnerVolumeSpecName "kube-api-access-mbl8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:12:43.967746 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.967717    2557 generic.go:356] "Generic (PLEG): container finished" podID="9d5eb3dd-a73f-4da4-8c97-403b1304dad8" containerID="99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0" exitCode=0
Sep 10 21:12:43.967878 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.967762    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"
Sep 10 21:12:43.967878 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.967816    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm" event={"ID":"9d5eb3dd-a73f-4da4-8c97-403b1304dad8","Type":"ContainerDied","Data":"99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0"}
Sep 10 21:12:43.967878 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.967876    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm" event={"ID":"9d5eb3dd-a73f-4da4-8c97-403b1304dad8","Type":"ContainerDied","Data":"5f335c4a1f5052e1aa48f5f5cd03e29da2711f8be388195f9ef7f93b5ae5770e"}
Sep 10 21:12:43.968056 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.967908    2557 scope.go:117] "RemoveContainer" containerID="99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0"
Sep 10 21:12:43.969301 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.969277    2557 generic.go:356] "Generic (PLEG): container finished" podID="ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" containerID="0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9" exitCode=0
Sep 10 21:12:43.969408 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.969318    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"
Sep 10 21:12:43.969408 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.969364    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76" event={"ID":"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d","Type":"ContainerDied","Data":"0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9"}
Sep 10 21:12:43.969534 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.969411    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76" event={"ID":"ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d","Type":"ContainerDied","Data":"d4dd755d9a389539b2997c22a21831c40ba2a8e0c339ebec09bb07d0460da06f"}
Sep 10 21:12:43.971934 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.971909    2557 generic.go:356] "Generic (PLEG): container finished" podID="cec4c795-89fc-4e9d-934f-b80f81a8db7c" containerID="1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968" exitCode=0
Sep 10 21:12:43.972052 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.971939    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"
Sep 10 21:12:43.972052 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.971942    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5" event={"ID":"cec4c795-89fc-4e9d-934f-b80f81a8db7c","Type":"ContainerDied","Data":"1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968"}
Sep 10 21:12:43.972052 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.972031    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5" event={"ID":"cec4c795-89fc-4e9d-934f-b80f81a8db7c","Type":"ContainerDied","Data":"813cfdb94b31c9e39671c809c8aec3571966c4f169794e058c5c06c615f997c5"}
Sep 10 21:12:43.977328 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.977296    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.977328 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.977323    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbl8f\" (UniqueName: \"kubernetes.io/projected/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d-kube-api-access-mbl8f\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.977455 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.977336    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4cnb\" (UniqueName: \"kubernetes.io/projected/cec4c795-89fc-4e9d-934f-b80f81a8db7c-kube-api-access-t4cnb\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.977455 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.977349    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cec4c795-89fc-4e9d-934f-b80f81a8db7c-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:12:43.978654 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.978634    2557 scope.go:117] "RemoveContainer" containerID="99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0"
Sep 10 21:12:43.979002 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:43.978976    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0\": container with ID starting with 99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0 not found: ID does not exist" containerID="99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0"
Sep 10 21:12:43.979058 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.979015    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0"} err="failed to get container status \"99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0\": rpc error: code = NotFound desc = could not find container \"99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0\": container with ID starting with 99ef41941b43c7e88ea3f44b35429ecca507303b751a37fc7e6f64249a3f98a0 not found: ID does not exist"
Sep 10 21:12:43.979058 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.979040    2557 scope.go:117] "RemoveContainer" containerID="0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9"
Sep 10 21:12:43.987952 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.987934    2557 scope.go:117] "RemoveContainer" containerID="0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9"
Sep 10 21:12:43.988236 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:43.988217    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9\": container with ID starting with 0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9 not found: ID does not exist" containerID="0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9"
Sep 10 21:12:43.988291 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.988243    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9"} err="failed to get container status \"0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9\": rpc error: code = NotFound desc = could not find container \"0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9\": container with ID starting with 0931cd5fb3d0ebcee35244b98225d4bfb71a2387cbef07aed94486448771faa9 not found: ID does not exist"
Sep 10 21:12:43.988291 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.988260    2557 scope.go:117] "RemoveContainer" containerID="1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968"
Sep 10 21:12:43.994518 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.994498    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"]
Sep 10 21:12:43.996907 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.996888    2557 scope.go:117] "RemoveContainer" containerID="1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968"
Sep 10 21:12:43.997163 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:12:43.997138    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968\": container with ID starting with 1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968 not found: ID does not exist" containerID="1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968"
Sep 10 21:12:43.997246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.997165    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968"} err="failed to get container status \"1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968\": rpc error: code = NotFound desc = could not find container \"1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968\": container with ID starting with 1f9446a2da051947f3f2758e71af20dfe3bb06ae60fda56f08390e64459b5968 not found: ID does not exist"
Sep 10 21:12:43.998328 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:43.998308    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-0d892491-d8599d944-9pnzm"]
Sep 10 21:12:44.008229 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.008205    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"]
Sep 10 21:12:44.014158 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.014139    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-0d892491-7c4995f89-69t76"]
Sep 10 21:12:44.025309 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.025284    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"]
Sep 10 21:12:44.030788 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.030769    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-0d892491-665759ffd6-qpsq5"]
Sep 10 21:12:44.036956 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.036935    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:12:44.586467 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.586435    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72634e55-dc83-4bc7-9185-d3fc4f50bb9c" path="/var/lib/kubelet/pods/72634e55-dc83-4bc7-9185-d3fc4f50bb9c/volumes"
Sep 10 21:12:44.586825 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.586811    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5eb3dd-a73f-4da4-8c97-403b1304dad8" path="/var/lib/kubelet/pods/9d5eb3dd-a73f-4da4-8c97-403b1304dad8/volumes"
Sep 10 21:12:44.587119 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.587108    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" path="/var/lib/kubelet/pods/ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d/volumes"
Sep 10 21:12:44.587717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.587702    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9d4c5e5-467f-4678-802d-5d500e1ec283" path="/var/lib/kubelet/pods/c9d4c5e5-467f-4678-802d-5d500e1ec283/volumes"
Sep 10 21:12:44.588029 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:44.588017    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cec4c795-89fc-4e9d-934f-b80f81a8db7c" path="/var/lib/kubelet/pods/cec4c795-89fc-4e9d-934f-b80f81a8db7c/volumes"
Sep 10 21:12:54.331011 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.330972    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"]
Sep 10 21:12:54.331612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331589    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" containerName="payload-processing"
Sep 10 21:12:54.331677 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331615    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" containerName="payload-processing"
Sep 10 21:12:54.331677 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331633    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9d4c5e5-467f-4678-802d-5d500e1ec283" containerName="payload-pre-processing"
Sep 10 21:12:54.331677 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331643    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9d4c5e5-467f-4678-802d-5d500e1ec283" containerName="payload-pre-processing"
Sep 10 21:12:54.331677 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331665    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5eb3dd-a73f-4da4-8c97-403b1304dad8" containerName="maas-api"
Sep 10 21:12:54.331677 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331674    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5eb3dd-a73f-4da4-8c97-403b1304dad8" containerName="maas-api"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331705    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72634e55-dc83-4bc7-9185-d3fc4f50bb9c" containerName="payload-processing"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331713    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="72634e55-dc83-4bc7-9185-d3fc4f50bb9c" containerName="payload-processing"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331726    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cec4c795-89fc-4e9d-934f-b80f81a8db7c" containerName="payload-pre-processing"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331735    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cec4c795-89fc-4e9d-934f-b80f81a8db7c" containerName="payload-pre-processing"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331837    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab8aeb7c-00c8-467a-955f-4bf5da0ebf8d" containerName="payload-processing"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331847    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9d4c5e5-467f-4678-802d-5d500e1ec283" containerName="payload-pre-processing"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331857    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d5eb3dd-a73f-4da4-8c97-403b1304dad8" containerName="maas-api"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331866    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="72634e55-dc83-4bc7-9185-d3fc4f50bb9c" containerName="payload-processing"
Sep 10 21:12:54.331978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.331877    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cec4c795-89fc-4e9d-934f-b80f81a8db7c" containerName="payload-pre-processing"
Sep 10 21:12:54.336837 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.336813    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.339818 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.339608    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-ea71f9-serving-cert\""
Sep 10 21:12:54.341654 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.341620    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:12:54.342678 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.342368    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:12:54.345509 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.343379    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:12:54.345509 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.344142    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:12:54.346513 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.346492    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"]
Sep 10 21:12:54.358833 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.358800    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-metrics-certs\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.358963 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.358906    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb6e6eb-321b-4301-bf64-05192737f04a-openshift-service-ca\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.359023 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.358960    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcn6n\" (UniqueName: \"kubernetes.io/projected/1eb6e6eb-321b-4301-bf64-05192737f04a-kube-api-access-qcn6n\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.359023 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.359010    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-maas-api-tls\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.459695 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.459657    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-metrics-certs\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.459890 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.459737    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb6e6eb-321b-4301-bf64-05192737f04a-openshift-service-ca\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.459890 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.459764    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcn6n\" (UniqueName: \"kubernetes.io/projected/1eb6e6eb-321b-4301-bf64-05192737f04a-kube-api-access-qcn6n\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.459890 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.459785    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-maas-api-tls\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.460422 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.460399    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb6e6eb-321b-4301-bf64-05192737f04a-openshift-service-ca\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.462166 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.462146    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-metrics-certs\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.462259 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.462239    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-maas-api-tls\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.470073 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.470041    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcn6n\" (UniqueName: \"kubernetes.io/projected/1eb6e6eb-321b-4301-bf64-05192737f04a-kube-api-access-qcn6n\") pod \"maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.652733 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.652632    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:54.931422 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:54.931394    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"]
Sep 10 21:12:54.932456 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:12:54.932431    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eb6e6eb_321b_4301_bf64_05192737f04a.slice/crio-c50b3b30489705637652c2c2955196de199e56b9ca4eebe886b919c26c861e5c WatchSource:0}: Error finding container c50b3b30489705637652c2c2955196de199e56b9ca4eebe886b919c26c861e5c: Status 404 returned error can't find the container with id c50b3b30489705637652c2c2955196de199e56b9ca4eebe886b919c26c861e5c
Sep 10 21:12:55.017051 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.017019    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"]
Sep 10 21:12:55.021642 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.021626    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.024671 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.024639    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-ea71f9-dockercfg-sknpm\""
Sep 10 21:12:55.024671 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.024646    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-ea71f9\""
Sep 10 21:12:55.033647 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.033622    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"]
Sep 10 21:12:55.041137 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.041113    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k" event={"ID":"1eb6e6eb-321b-4301-bf64-05192737f04a","Type":"ContainerStarted","Data":"c50b3b30489705637652c2c2955196de199e56b9ca4eebe886b919c26c861e5c"}
Sep 10 21:12:55.063108 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.063076    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08291718-342f-479c-9fa3-7ce9847f50a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.063225 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.063134    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25v2t\" (UniqueName: \"kubernetes.io/projected/08291718-342f-479c-9fa3-7ce9847f50a6-kube-api-access-25v2t\") pod \"payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.163814 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.163770    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25v2t\" (UniqueName: \"kubernetes.io/projected/08291718-342f-479c-9fa3-7ce9847f50a6-kube-api-access-25v2t\") pod \"payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.163998 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.163865    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08291718-342f-479c-9fa3-7ce9847f50a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.164571 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.164550    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08291718-342f-479c-9fa3-7ce9847f50a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.171923 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.171893    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25v2t\" (UniqueName: \"kubernetes.io/projected/08291718-342f-479c-9fa3-7ce9847f50a6-kube-api-access-25v2t\") pod \"payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.333431 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.333392    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:55.605386 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:55.605354    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"]
Sep 10 21:12:55.607301 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:12:55.607268    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08291718_342f_479c_9fa3_7ce9847f50a6.slice/crio-1838e622a99fe7eea62553aa541f1a8b576991aa341591de39eb83af63281fc4 WatchSource:0}: Error finding container 1838e622a99fe7eea62553aa541f1a8b576991aa341591de39eb83af63281fc4: Status 404 returned error can't find the container with id 1838e622a99fe7eea62553aa541f1a8b576991aa341591de39eb83af63281fc4
Sep 10 21:12:56.048338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:56.048301    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx" event={"ID":"08291718-342f-479c-9fa3-7ce9847f50a6","Type":"ContainerStarted","Data":"cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f"}
Sep 10 21:12:56.048535 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:56.048368    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:12:56.048535 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:56.048395    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx" event={"ID":"08291718-342f-479c-9fa3-7ce9847f50a6","Type":"ContainerStarted","Data":"1838e622a99fe7eea62553aa541f1a8b576991aa341591de39eb83af63281fc4"}
Sep 10 21:12:56.049901 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:56.049873    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k" event={"ID":"1eb6e6eb-321b-4301-bf64-05192737f04a","Type":"ContainerStarted","Data":"6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af"}
Sep 10 21:12:56.050052 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:56.049927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:12:56.067198 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:56.067143    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx" podStartSLOduration=2.06712952 podStartE2EDuration="2.06712952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:56.065135533 +0000 UTC m=+856.084540852" watchObservedRunningTime="2026-09-10 21:12:56.06712952 +0000 UTC m=+856.086534837"
Sep 10 21:12:56.085042 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:56.084996    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k" podStartSLOduration=1.618583938 podStartE2EDuration="2.084982282s" totalImagesPullingTime="466.398344ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:12:56.083885404 +0000 UTC m=+856.103290721" watchObservedRunningTime="2026-09-10 21:12:56.084982282 +0000 UTC m=+856.104387638"
Sep 10 21:12:59.544612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:12:59.544576    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:13:02.067613 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:02.067585    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:13:07.064495 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:07.064463    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:13:16.940041 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:16.940003    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:13:24.837045 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.837013    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"]
Sep 10 21:13:24.840612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.840592    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:24.843272 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.843248    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ea71f9\"/\"kube-root-ca.crt\""
Sep 10 21:13:24.844161 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.844142    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ea71f9\"/\"default-dockercfg-zjvgg\""
Sep 10 21:13:24.844593 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.844578    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-ea71f9\"/\"explicit-model-ea71f9-kserve-self-signed-certs\""
Sep 10 21:13:24.844642 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.844587    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-ea71f9\"/\"openshift-service-ca.crt\""
Sep 10 21:13:24.850890 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.850868    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"]
Sep 10 21:13:24.919641 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.919606    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-home\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:24.919853 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.919674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-tmp-dir\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:24.919853 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.919775    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac12776-61ee-4459-b441-5268fec3ef02-tls-certs\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:24.919853 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.919810    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mj6j\" (UniqueName: \"kubernetes.io/projected/bac12776-61ee-4459-b441-5268fec3ef02-kube-api-access-5mj6j\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:24.920030 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.919896    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-dshm\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:24.920030 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:24.919920    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-model-cache\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.020785 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.020745    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac12776-61ee-4459-b441-5268fec3ef02-tls-certs\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.020961 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.020799    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mj6j\" (UniqueName: \"kubernetes.io/projected/bac12776-61ee-4459-b441-5268fec3ef02-kube-api-access-5mj6j\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.020961 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.020867    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-dshm\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.020961 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.020882    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-model-cache\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.020961 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.020926    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-home\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.021189 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.020979    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-tmp-dir\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.021416 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.021386    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-model-cache\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.021531 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.021426    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-tmp-dir\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.021531 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.021397    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-home\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.023043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.023021    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-dshm\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.023318 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.023302    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac12776-61ee-4459-b441-5268fec3ef02-tls-certs\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.029597 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.029573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mj6j\" (UniqueName: \"kubernetes.io/projected/bac12776-61ee-4459-b441-5268fec3ef02-kube-api-access-5mj6j\") pod \"explicit-model-ea71f9-kserve-7f9557fcf9-n59p5\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") " pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.150812 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.150732    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:25.429883 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:25.429857    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"]
Sep 10 21:13:25.431704 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:13:25.431661    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbac12776_61ee_4459_b441_5268fec3ef02.slice/crio-129e64309aca8dfa52f42411749dc1cc3f44e7d5af6e464835452dd12768e945 WatchSource:0}: Error finding container 129e64309aca8dfa52f42411749dc1cc3f44e7d5af6e464835452dd12768e945: Status 404 returned error can't find the container with id 129e64309aca8dfa52f42411749dc1cc3f44e7d5af6e464835452dd12768e945
Sep 10 21:13:26.241791 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:26.241750    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5" event={"ID":"bac12776-61ee-4459-b441-5268fec3ef02","Type":"ContainerStarted","Data":"c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8"}
Sep 10 21:13:26.242167 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:26.241803    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5" event={"ID":"bac12776-61ee-4459-b441-5268fec3ef02","Type":"ContainerStarted","Data":"129e64309aca8dfa52f42411749dc1cc3f44e7d5af6e464835452dd12768e945"}
Sep 10 21:13:26.261927 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:26.261883    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5" podStartSLOduration=2.261868077 podStartE2EDuration="2.261868077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:26.260050152 +0000 UTC m=+886.279455469" watchObservedRunningTime="2026-09-10 21:13:26.261868077 +0000 UTC m=+886.281273395"
Sep 10 21:13:30.618335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:30.618300    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"]
Sep 10 21:13:32.847489 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:32.847397    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:13:35.150847 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:35.150799    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:40.431993 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:40.431959    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:13:40.434344 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:40.434324    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:13:41.423465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.423435    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"]
Sep 10 21:13:41.423678 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.423634    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k" podUID="1eb6e6eb-321b-4301-bf64-05192737f04a" containerName="maas-api" containerID="cri-o://6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af" gracePeriod=30
Sep 10 21:13:41.511960 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.510558    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"]
Sep 10 21:13:41.511960 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.510906    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx" podUID="08291718-342f-479c-9fa3-7ce9847f50a6" containerName="payload-pre-processing" containerID="cri-o://cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f" gracePeriod=30
Sep 10 21:13:41.764025 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.764001    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:13:41.791770 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.791730    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:13:41.873298 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873250    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-metrics-certs\") pod \"1eb6e6eb-321b-4301-bf64-05192737f04a\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") "
Sep 10 21:13:41.873298 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873302    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcn6n\" (UniqueName: \"kubernetes.io/projected/1eb6e6eb-321b-4301-bf64-05192737f04a-kube-api-access-qcn6n\") pod \"1eb6e6eb-321b-4301-bf64-05192737f04a\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") "
Sep 10 21:13:41.873592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873393    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb6e6eb-321b-4301-bf64-05192737f04a-openshift-service-ca\") pod \"1eb6e6eb-321b-4301-bf64-05192737f04a\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") "
Sep 10 21:13:41.873592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873424    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-maas-api-tls\") pod \"1eb6e6eb-321b-4301-bf64-05192737f04a\" (UID: \"1eb6e6eb-321b-4301-bf64-05192737f04a\") "
Sep 10 21:13:41.873592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873448    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08291718-342f-479c-9fa3-7ce9847f50a6-plugins-config-volume\") pod \"08291718-342f-479c-9fa3-7ce9847f50a6\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") "
Sep 10 21:13:41.873592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873503    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25v2t\" (UniqueName: \"kubernetes.io/projected/08291718-342f-479c-9fa3-7ce9847f50a6-kube-api-access-25v2t\") pod \"08291718-342f-479c-9fa3-7ce9847f50a6\" (UID: \"08291718-342f-479c-9fa3-7ce9847f50a6\") "
Sep 10 21:13:41.873886 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873860    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eb6e6eb-321b-4301-bf64-05192737f04a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1eb6e6eb-321b-4301-bf64-05192737f04a" (UID: "1eb6e6eb-321b-4301-bf64-05192737f04a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:41.873942 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.873909    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08291718-342f-479c-9fa3-7ce9847f50a6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "08291718-342f-479c-9fa3-7ce9847f50a6" (UID: "08291718-342f-479c-9fa3-7ce9847f50a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:13:41.875513 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.875487    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1eb6e6eb-321b-4301-bf64-05192737f04a" (UID: "1eb6e6eb-321b-4301-bf64-05192737f04a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:41.875604 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.875522    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eb6e6eb-321b-4301-bf64-05192737f04a-kube-api-access-qcn6n" (OuterVolumeSpecName: "kube-api-access-qcn6n") pod "1eb6e6eb-321b-4301-bf64-05192737f04a" (UID: "1eb6e6eb-321b-4301-bf64-05192737f04a"). InnerVolumeSpecName "kube-api-access-qcn6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:41.875910 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.875883    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08291718-342f-479c-9fa3-7ce9847f50a6-kube-api-access-25v2t" (OuterVolumeSpecName: "kube-api-access-25v2t") pod "08291718-342f-479c-9fa3-7ce9847f50a6" (UID: "08291718-342f-479c-9fa3-7ce9847f50a6"). InnerVolumeSpecName "kube-api-access-25v2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:41.875999 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.875910    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1eb6e6eb-321b-4301-bf64-05192737f04a" (UID: "1eb6e6eb-321b-4301-bf64-05192737f04a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:41.975080 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.975046    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1eb6e6eb-321b-4301-bf64-05192737f04a-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:41.975080 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.975078    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:41.975280 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.975094    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08291718-342f-479c-9fa3-7ce9847f50a6-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:41.975280 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.975104    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25v2t\" (UniqueName: \"kubernetes.io/projected/08291718-342f-479c-9fa3-7ce9847f50a6-kube-api-access-25v2t\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:41.975280 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.975113    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1eb6e6eb-321b-4301-bf64-05192737f04a-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:41.975280 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:41.975121    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcn6n\" (UniqueName: \"kubernetes.io/projected/1eb6e6eb-321b-4301-bf64-05192737f04a-kube-api-access-qcn6n\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:42.335407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.335322    2557 generic.go:356] "Generic (PLEG): container finished" podID="08291718-342f-479c-9fa3-7ce9847f50a6" containerID="cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f" exitCode=0
Sep 10 21:13:42.335407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.335366    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"
Sep 10 21:13:42.335628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.335401    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx" event={"ID":"08291718-342f-479c-9fa3-7ce9847f50a6","Type":"ContainerDied","Data":"cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f"}
Sep 10 21:13:42.335628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.335456    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx" event={"ID":"08291718-342f-479c-9fa3-7ce9847f50a6","Type":"ContainerDied","Data":"1838e622a99fe7eea62553aa541f1a8b576991aa341591de39eb83af63281fc4"}
Sep 10 21:13:42.335628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.335478    2557 scope.go:117] "RemoveContainer" containerID="cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f"
Sep 10 21:13:42.336848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.336827    2557 generic.go:356] "Generic (PLEG): container finished" podID="1eb6e6eb-321b-4301-bf64-05192737f04a" containerID="6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af" exitCode=0
Sep 10 21:13:42.336995 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.336868    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"
Sep 10 21:13:42.336995 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.336857    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k" event={"ID":"1eb6e6eb-321b-4301-bf64-05192737f04a","Type":"ContainerDied","Data":"6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af"}
Sep 10 21:13:42.336995 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.336925    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k" event={"ID":"1eb6e6eb-321b-4301-bf64-05192737f04a","Type":"ContainerDied","Data":"c50b3b30489705637652c2c2955196de199e56b9ca4eebe886b919c26c861e5c"}
Sep 10 21:13:42.348768 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.348750    2557 scope.go:117] "RemoveContainer" containerID="cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f"
Sep 10 21:13:42.349041 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:13:42.349023    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f\": container with ID starting with cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f not found: ID does not exist" containerID="cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f"
Sep 10 21:13:42.349109 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.349047    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f"} err="failed to get container status \"cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f\": rpc error: code = NotFound desc = could not find container \"cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f\": container with ID starting with cd1c6de5e25198431c411efcbb0d63769201e2e7e125c09b92dfad3efd65ff4f not found: ID does not exist"
Sep 10 21:13:42.349109 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.349063    2557 scope.go:117] "RemoveContainer" containerID="6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af"
Sep 10 21:13:42.357098 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.357079    2557 scope.go:117] "RemoveContainer" containerID="6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af"
Sep 10 21:13:42.357373 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:13:42.357355    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af\": container with ID starting with 6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af not found: ID does not exist" containerID="6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af"
Sep 10 21:13:42.357420 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.357379    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af"} err="failed to get container status \"6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af\": rpc error: code = NotFound desc = could not find container \"6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af\": container with ID starting with 6e696445d938d28ec037654c5cc61abd7f1efd7e48332fb0821be31d0d1c66af not found: ID does not exist"
Sep 10 21:13:42.362542 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.362519    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"]
Sep 10 21:13:42.366504 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.366481    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-ea71f9-6db85bccd-gjqtx"]
Sep 10 21:13:42.377504 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.377481    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"]
Sep 10 21:13:42.381897 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.381873    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ea71f9-746c46b96c-gc95k"]
Sep 10 21:13:42.586938 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.586861    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08291718-342f-479c-9fa3-7ce9847f50a6" path="/var/lib/kubelet/pods/08291718-342f-479c-9fa3-7ce9847f50a6/volumes"
Sep 10 21:13:42.587338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:42.587319    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eb6e6eb-321b-4301-bf64-05192737f04a" path="/var/lib/kubelet/pods/1eb6e6eb-321b-4301-bf64-05192737f04a/volumes"
Sep 10 21:13:45.637949 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:45.637889    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5" podUID="bac12776-61ee-4459-b441-5268fec3ef02" containerName="main" containerID="cri-o://c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8" gracePeriod=45
Sep 10 21:13:45.884055 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:45.884030    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:46.008796 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.008763    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mj6j\" (UniqueName: \"kubernetes.io/projected/bac12776-61ee-4459-b441-5268fec3ef02-kube-api-access-5mj6j\") pod \"bac12776-61ee-4459-b441-5268fec3ef02\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") "
Sep 10 21:13:46.008959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.008805    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-model-cache\") pod \"bac12776-61ee-4459-b441-5268fec3ef02\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") "
Sep 10 21:13:46.008959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.008857    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-home\") pod \"bac12776-61ee-4459-b441-5268fec3ef02\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") "
Sep 10 21:13:46.008959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.008890    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac12776-61ee-4459-b441-5268fec3ef02-tls-certs\") pod \"bac12776-61ee-4459-b441-5268fec3ef02\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") "
Sep 10 21:13:46.008959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.008926    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-tmp-dir\") pod \"bac12776-61ee-4459-b441-5268fec3ef02\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") "
Sep 10 21:13:46.008959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.008940    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-dshm\") pod \"bac12776-61ee-4459-b441-5268fec3ef02\" (UID: \"bac12776-61ee-4459-b441-5268fec3ef02\") "
Sep 10 21:13:46.009223 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.009143    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-home" (OuterVolumeSpecName: "home") pod "bac12776-61ee-4459-b441-5268fec3ef02" (UID: "bac12776-61ee-4459-b441-5268fec3ef02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:46.009223 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.009156    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-model-cache" (OuterVolumeSpecName: "model-cache") pod "bac12776-61ee-4459-b441-5268fec3ef02" (UID: "bac12776-61ee-4459-b441-5268fec3ef02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:46.009223 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.009196    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bac12776-61ee-4459-b441-5268fec3ef02" (UID: "bac12776-61ee-4459-b441-5268fec3ef02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:46.011096 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.011065    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-dshm" (OuterVolumeSpecName: "dshm") pod "bac12776-61ee-4459-b441-5268fec3ef02" (UID: "bac12776-61ee-4459-b441-5268fec3ef02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:13:46.011096 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.011082    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac12776-61ee-4459-b441-5268fec3ef02-kube-api-access-5mj6j" (OuterVolumeSpecName: "kube-api-access-5mj6j") pod "bac12776-61ee-4459-b441-5268fec3ef02" (UID: "bac12776-61ee-4459-b441-5268fec3ef02"). InnerVolumeSpecName "kube-api-access-5mj6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:13:46.011096 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.011081    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac12776-61ee-4459-b441-5268fec3ef02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bac12776-61ee-4459-b441-5268fec3ef02" (UID: "bac12776-61ee-4459-b441-5268fec3ef02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:13:46.110106 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.110078    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mj6j\" (UniqueName: \"kubernetes.io/projected/bac12776-61ee-4459-b441-5268fec3ef02-kube-api-access-5mj6j\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:46.110106 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.110102    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:46.110106 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.110113    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:46.110300 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.110121    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac12776-61ee-4459-b441-5268fec3ef02-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:46.110300 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.110129    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:46.110300 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.110137    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac12776-61ee-4459-b441-5268fec3ef02-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:13:46.363044 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.362959    2557 generic.go:356] "Generic (PLEG): container finished" podID="bac12776-61ee-4459-b441-5268fec3ef02" containerID="c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8" exitCode=0
Sep 10 21:13:46.363044 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.362998    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"
Sep 10 21:13:46.363044 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.363022    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5" event={"ID":"bac12776-61ee-4459-b441-5268fec3ef02","Type":"ContainerDied","Data":"c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8"}
Sep 10 21:13:46.363335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.363053    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5" event={"ID":"bac12776-61ee-4459-b441-5268fec3ef02","Type":"ContainerDied","Data":"129e64309aca8dfa52f42411749dc1cc3f44e7d5af6e464835452dd12768e945"}
Sep 10 21:13:46.363335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.363071    2557 scope.go:117] "RemoveContainer" containerID="c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8"
Sep 10 21:13:46.372346 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.372327    2557 scope.go:117] "RemoveContainer" containerID="c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8"
Sep 10 21:13:46.372602 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:13:46.372584    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8\": container with ID starting with c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8 not found: ID does not exist" containerID="c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8"
Sep 10 21:13:46.372650 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.372612    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8"} err="failed to get container status \"c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8\": rpc error: code = NotFound desc = could not find container \"c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8\": container with ID starting with c2876c05aa09c51e376f7fc885eeb17e906ab9ed64cec7926aedbedb2e4caba8 not found: ID does not exist"
Sep 10 21:13:46.384833 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.384802    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"]
Sep 10 21:13:46.391279 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.391252    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-ea71f9/explicit-model-ea71f9-kserve-7f9557fcf9-n59p5"]
Sep 10 21:13:46.585637 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:46.585605    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac12776-61ee-4459-b441-5268fec3ef02" path="/var/lib/kubelet/pods/bac12776-61ee-4459-b441-5268fec3ef02/volumes"
Sep 10 21:13:50.540130 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:50.540092    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:13:57.144871 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.144836    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"]
Sep 10 21:13:57.145246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145203    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac12776-61ee-4459-b441-5268fec3ef02" containerName="main"
Sep 10 21:13:57.145246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145215    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac12776-61ee-4459-b441-5268fec3ef02" containerName="main"
Sep 10 21:13:57.145246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145226    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eb6e6eb-321b-4301-bf64-05192737f04a" containerName="maas-api"
Sep 10 21:13:57.145246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145232    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eb6e6eb-321b-4301-bf64-05192737f04a" containerName="maas-api"
Sep 10 21:13:57.145246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145239    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08291718-342f-479c-9fa3-7ce9847f50a6" containerName="payload-pre-processing"
Sep 10 21:13:57.145246 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145245    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="08291718-342f-479c-9fa3-7ce9847f50a6" containerName="payload-pre-processing"
Sep 10 21:13:57.145438 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145309    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eb6e6eb-321b-4301-bf64-05192737f04a" containerName="maas-api"
Sep 10 21:13:57.145438 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145317    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac12776-61ee-4459-b441-5268fec3ef02" containerName="main"
Sep 10 21:13:57.145438 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.145323    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="08291718-342f-479c-9fa3-7ce9847f50a6" containerName="payload-pre-processing"
Sep 10 21:13:57.149713 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.149675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.155093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.154901    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-21de0f\"/\"openshift-service-ca.crt\""
Sep 10 21:13:57.155093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.154949    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-21de0f\"/\"noten-model-21de0f-kserve-self-signed-certs\""
Sep 10 21:13:57.155093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.154958    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-21de0f\"/\"default-dockercfg-l99j6\""
Sep 10 21:13:57.155093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.154958    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-21de0f\"/\"kube-root-ca.crt\""
Sep 10 21:13:57.171717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.168479    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"]
Sep 10 21:13:57.200840 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.200811    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-model-cache\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.200989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.200843    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbq5f\" (UniqueName: \"kubernetes.io/projected/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-kube-api-access-zbq5f\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.200989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.200863    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-home\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.200989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.200947    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-dshm\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.200989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.200972    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tls-certs\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.201151 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.201007    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tmp-dir\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.302433    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-model-cache\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.302490    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbq5f\" (UniqueName: \"kubernetes.io/projected/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-kube-api-access-zbq5f\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.302521    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-home\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.302572    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-dshm\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.302597    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tls-certs\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.302648    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tmp-dir\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.302989    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-model-cache\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.302476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.303022    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tmp-dir\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.303620 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.303233    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-home\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.305652 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.305633    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-dshm\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.306050 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.306033    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tls-certs\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.314118 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.314089    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbq5f\" (UniqueName: \"kubernetes.io/projected/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-kube-api-access-zbq5f\") pod \"noten-model-21de0f-kserve-6c7ccd4b7-5xcc5\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") " pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.459842 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.459754    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:13:57.737113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:57.737086    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"]
Sep 10 21:13:57.738805 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:13:57.738773    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdeffd68_1688_47c6_a34f_0b8ec9c8cf7c.slice/crio-9fe50fcb595cc50573f43dee21edcdf55c76ce36e4ee2cbcbe33c261d26a11c0 WatchSource:0}: Error finding container 9fe50fcb595cc50573f43dee21edcdf55c76ce36e4ee2cbcbe33c261d26a11c0: Status 404 returned error can't find the container with id 9fe50fcb595cc50573f43dee21edcdf55c76ce36e4ee2cbcbe33c261d26a11c0
Sep 10 21:13:58.431989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:58.431955    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5" event={"ID":"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c","Type":"ContainerStarted","Data":"9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a"}
Sep 10 21:13:58.431989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:58.431995    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5" event={"ID":"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c","Type":"ContainerStarted","Data":"9fe50fcb595cc50573f43dee21edcdf55c76ce36e4ee2cbcbe33c261d26a11c0"}
Sep 10 21:13:58.453593 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:13:58.453545    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5" podStartSLOduration=1.453531387 podStartE2EDuration="1.453531387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:13:58.451905624 +0000 UTC m=+918.471310943" watchObservedRunningTime="2026-09-10 21:13:58.453531387 +0000 UTC m=+918.472936705"
Sep 10 21:14:01.232286 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:01.232245    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"]
Sep 10 21:14:04.539235 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:04.539198    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:14:07.460368 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:07.460337    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:14:16.251254 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.251211    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5" podUID="fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" containerName="main" containerID="cri-o://9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a" gracePeriod=45
Sep 10 21:14:16.502317 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.502252    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:14:16.529303 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.529268    2557 generic.go:356] "Generic (PLEG): container finished" podID="fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" containerID="9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a" exitCode=0
Sep 10 21:14:16.529303 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.529306    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"
Sep 10 21:14:16.529507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.529343    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5" event={"ID":"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c","Type":"ContainerDied","Data":"9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a"}
Sep 10 21:14:16.529507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.529390    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5" event={"ID":"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c","Type":"ContainerDied","Data":"9fe50fcb595cc50573f43dee21edcdf55c76ce36e4ee2cbcbe33c261d26a11c0"}
Sep 10 21:14:16.529507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.529410    2557 scope.go:117] "RemoveContainer" containerID="9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a"
Sep 10 21:14:16.537849 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.537829    2557 scope.go:117] "RemoveContainer" containerID="9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a"
Sep 10 21:14:16.538101 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:14:16.538081    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a\": container with ID starting with 9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a not found: ID does not exist" containerID="9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a"
Sep 10 21:14:16.538190 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.538107    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a"} err="failed to get container status \"9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a\": rpc error: code = NotFound desc = could not find container \"9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a\": container with ID starting with 9c652c4fbf8cc3a240bbc646a750e96324ef798acf7adb80ba6dd1a5bca0c75a not found: ID does not exist"
Sep 10 21:14:16.565900 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.565877    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tmp-dir\") pod \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") "
Sep 10 21:14:16.565989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.565937    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-model-cache\") pod \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") "
Sep 10 21:14:16.565989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.565978    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tls-certs\") pod \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") "
Sep 10 21:14:16.566062 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566029    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-home\") pod \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") "
Sep 10 21:14:16.566097 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566061    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-dshm\") pod \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") "
Sep 10 21:14:16.566138 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566117    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbq5f\" (UniqueName: \"kubernetes.io/projected/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-kube-api-access-zbq5f\") pod \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\" (UID: \"fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c\") "
Sep 10 21:14:16.566183 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566136    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" (UID: "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:16.566237 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566213    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-model-cache" (OuterVolumeSpecName: "model-cache") pod "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" (UID: "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:16.566308 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566289    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-home" (OuterVolumeSpecName: "home") pod "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" (UID: "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:16.566378 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566364    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:16.566441 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566380    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:16.566441 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.566392    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:16.568154 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.568133    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-kube-api-access-zbq5f" (OuterVolumeSpecName: "kube-api-access-zbq5f") pod "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" (UID: "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c"). InnerVolumeSpecName "kube-api-access-zbq5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:16.568241 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.568196    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" (UID: "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:14:16.568241 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.568206    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-dshm" (OuterVolumeSpecName: "dshm") pod "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" (UID: "fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:14:16.667230 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.667204    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbq5f\" (UniqueName: \"kubernetes.io/projected/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-kube-api-access-zbq5f\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:16.667230 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.667229    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:16.667409 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.667243    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:16.846733 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.846662    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"]
Sep 10 21:14:16.856343 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:16.856313    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-21de0f/noten-model-21de0f-kserve-6c7ccd4b7-5xcc5"]
Sep 10 21:14:18.584700 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:18.584654    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" path="/var/lib/kubelet/pods/fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c/volumes"
Sep 10 21:14:25.242165 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.242129    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"]
Sep 10 21:14:25.242623 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.242607    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" containerName="main"
Sep 10 21:14:25.242691 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.242624    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" containerName="main"
Sep 10 21:14:25.242732 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.242720    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdeffd68-1688-47c6-a34f-0b8ec9c8cf7c" containerName="main"
Sep 10 21:14:25.247213 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.247196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"
Sep 10 21:14:25.250273 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.250254    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8q9zm\""
Sep 10 21:14:25.250368 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.250258    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:14:25.256736 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.256713    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"]
Sep 10 21:14:25.343506 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.343464    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/105d5a8c-745a-400b-ac95-f91bd2647737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh\" (UID: \"105d5a8c-745a-400b-ac95-f91bd2647737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"
Sep 10 21:14:25.444253 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.444224    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/105d5a8c-745a-400b-ac95-f91bd2647737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh\" (UID: \"105d5a8c-745a-400b-ac95-f91bd2647737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"
Sep 10 21:14:25.444889 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.444867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/105d5a8c-745a-400b-ac95-f91bd2647737-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh\" (UID: \"105d5a8c-745a-400b-ac95-f91bd2647737\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"
Sep 10 21:14:25.558781 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.558722    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"
Sep 10 21:14:25.834887 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:25.834860    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"]
Sep 10 21:14:25.835879 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:14:25.835851    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105d5a8c_745a_400b_ac95_f91bd2647737.slice/crio-e259e978c715ef8151bd0dc24f20ccb715a0b063b2e2574154c05ae60c113b41 WatchSource:0}: Error finding container e259e978c715ef8151bd0dc24f20ccb715a0b063b2e2574154c05ae60c113b41: Status 404 returned error can't find the container with id e259e978c715ef8151bd0dc24f20ccb715a0b063b2e2574154c05ae60c113b41
Sep 10 21:14:26.238202 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.238167    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"]
Sep 10 21:14:26.243418 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.243399    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.247925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.247773    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:14:26.247925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.247774    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c6e2e41d-serving-cert\""
Sep 10 21:14:26.247925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.247800    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:14:26.247925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.247822    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:14:26.257542 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.257515    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"]
Sep 10 21:14:26.352093 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.352061    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/508cb91d-f2ca-4fcb-8c13-943206a0e903-openshift-service-ca\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.352227 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.352102    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-maas-api-tls\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.352227 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.352119    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46hlc\" (UniqueName: \"kubernetes.io/projected/508cb91d-f2ca-4fcb-8c13-943206a0e903-kube-api-access-46hlc\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.352227 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.352183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-metrics-certs\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.453495 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.453465    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/508cb91d-f2ca-4fcb-8c13-943206a0e903-openshift-service-ca\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.453635 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.453507    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-maas-api-tls\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.453635 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.453527    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46hlc\" (UniqueName: \"kubernetes.io/projected/508cb91d-f2ca-4fcb-8c13-943206a0e903-kube-api-access-46hlc\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.453635 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.453574    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-metrics-certs\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.454133 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.454110    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/508cb91d-f2ca-4fcb-8c13-943206a0e903-openshift-service-ca\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.456108 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.456080    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-metrics-certs\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.456264 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.456247    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-maas-api-tls\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.463267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.463245    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46hlc\" (UniqueName: \"kubernetes.io/projected/508cb91d-f2ca-4fcb-8c13-943206a0e903-kube-api-access-46hlc\") pod \"maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.557891 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.557819    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:26.597248 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.597213    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh" event={"ID":"105d5a8c-745a-400b-ac95-f91bd2647737","Type":"ContainerStarted","Data":"e259e978c715ef8151bd0dc24f20ccb715a0b063b2e2574154c05ae60c113b41"}
Sep 10 21:14:26.843194 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.843154    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"]
Sep 10 21:14:26.844468 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:14:26.844432    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508cb91d_f2ca_4fcb_8c13_943206a0e903.slice/crio-f8a92b822c46250f4e22e0fdd195cbf0d53bd8c6838fd8f9e9a7eec2f79a8eee WatchSource:0}: Error finding container f8a92b822c46250f4e22e0fdd195cbf0d53bd8c6838fd8f9e9a7eec2f79a8eee: Status 404 returned error can't find the container with id f8a92b822c46250f4e22e0fdd195cbf0d53bd8c6838fd8f9e9a7eec2f79a8eee
Sep 10 21:14:26.960703 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.960654    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"]
Sep 10 21:14:26.964628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.964603    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:26.968927 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.968908    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c6e2e41d-dockercfg-z5mj7\""
Sep 10 21:14:26.969043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.968985    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c6e2e41d\""
Sep 10 21:14:26.976095 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:26.976036    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"]
Sep 10 21:14:27.059201 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.059164    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2frqv\" (UniqueName: \"kubernetes.io/projected/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-kube-api-access-2frqv\") pod \"payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") " pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:27.059377 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.059238    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") " pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:27.160437 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.160318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") " pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:27.160644 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.160539    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2frqv\" (UniqueName: \"kubernetes.io/projected/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-kube-api-access-2frqv\") pod \"payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") " pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:27.161101 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.161074    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-plugins-config-volume\") pod \"payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") " pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:27.173785 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.173753    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2frqv\" (UniqueName: \"kubernetes.io/projected/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-kube-api-access-2frqv\") pod \"payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") " pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:27.280442 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.280411    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:27.562972 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.562946    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"]
Sep 10 21:14:27.606286 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.606237    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt" event={"ID":"508cb91d-f2ca-4fcb-8c13-943206a0e903","Type":"ContainerStarted","Data":"816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20"}
Sep 10 21:14:27.606455 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.606335    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt" event={"ID":"508cb91d-f2ca-4fcb-8c13-943206a0e903","Type":"ContainerStarted","Data":"f8a92b822c46250f4e22e0fdd195cbf0d53bd8c6838fd8f9e9a7eec2f79a8eee"}
Sep 10 21:14:27.606455 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.606390    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:27.624676 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.624622    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt" podStartSLOduration=1.212186077 podStartE2EDuration="1.624607657s" totalImagesPullingTime="412.42158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:27.624265902 +0000 UTC m=+947.643671224" watchObservedRunningTime="2026-09-10 21:14:27.624607657 +0000 UTC m=+947.644012975"
Sep 10 21:14:27.937073 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:27.936982    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:14:28.140171 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:14:28.140142    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a83608c_81f1_40eb_b2d9_8d3ce76c6597.slice/crio-5987ef83fd8c1f2e02451806f91cd84949ac5479a9068e42a27e7013ab839766 WatchSource:0}: Error finding container 5987ef83fd8c1f2e02451806f91cd84949ac5479a9068e42a27e7013ab839766: Status 404 returned error can't find the container with id 5987ef83fd8c1f2e02451806f91cd84949ac5479a9068e42a27e7013ab839766
Sep 10 21:14:28.613276 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:28.613188    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2" event={"ID":"7a83608c-81f1-40eb-b2d9-8d3ce76c6597","Type":"ContainerStarted","Data":"6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4"}
Sep 10 21:14:28.613276 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:28.613251    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2" event={"ID":"7a83608c-81f1-40eb-b2d9-8d3ce76c6597","Type":"ContainerStarted","Data":"5987ef83fd8c1f2e02451806f91cd84949ac5479a9068e42a27e7013ab839766"}
Sep 10 21:14:28.615406 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:28.615366    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh" event={"ID":"105d5a8c-745a-400b-ac95-f91bd2647737","Type":"ContainerStarted","Data":"b3d019b9aa9bf4826e3bd14194e306c7b985ee77d4501d67e4403028754e9836"}
Sep 10 21:14:28.635075 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:28.635034    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2" podStartSLOduration=2.63502062 podStartE2EDuration="2.63502062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:28.633510318 +0000 UTC m=+948.652915649" watchObservedRunningTime="2026-09-10 21:14:28.63502062 +0000 UTC m=+948.654425979"
Sep 10 21:14:28.650829 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:28.650790    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh" podStartSLOduration=1.2826654579999999 podStartE2EDuration="3.650777846s" totalImagesPullingTime="2.368112388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:14:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:14:28.64930022 +0000 UTC m=+948.668705539" watchObservedRunningTime="2026-09-10 21:14:28.650777846 +0000 UTC m=+948.670183187"
Sep 10 21:14:29.623534 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:29.623496    2557 generic.go:356] "Generic (PLEG): container finished" podID="105d5a8c-745a-400b-ac95-f91bd2647737" containerID="b3d019b9aa9bf4826e3bd14194e306c7b985ee77d4501d67e4403028754e9836" exitCode=0
Sep 10 21:14:29.623996 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:29.623578    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh" event={"ID":"105d5a8c-745a-400b-ac95-f91bd2647737","Type":"ContainerDied","Data":"b3d019b9aa9bf4826e3bd14194e306c7b985ee77d4501d67e4403028754e9836"}
Sep 10 21:14:29.623996 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:29.623868    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:30.757399 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:30.757374    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"
Sep 10 21:14:30.897140 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:30.897069    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/105d5a8c-745a-400b-ac95-f91bd2647737-maas-api-service-ca\") pod \"105d5a8c-745a-400b-ac95-f91bd2647737\" (UID: \"105d5a8c-745a-400b-ac95-f91bd2647737\") "
Sep 10 21:14:30.897396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:30.897373    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105d5a8c-745a-400b-ac95-f91bd2647737-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "105d5a8c-745a-400b-ac95-f91bd2647737" (UID: "105d5a8c-745a-400b-ac95-f91bd2647737"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:30.998551 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:30.998519    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/105d5a8c-745a-400b-ac95-f91bd2647737-maas-api-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:31.635203 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:31.635163    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"
Sep 10 21:14:31.635375 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:31.635204    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh" event={"ID":"105d5a8c-745a-400b-ac95-f91bd2647737","Type":"ContainerDied","Data":"e259e978c715ef8151bd0dc24f20ccb715a0b063b2e2574154c05ae60c113b41"}
Sep 10 21:14:31.635375 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:31.635248    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e259e978c715ef8151bd0dc24f20ccb715a0b063b2e2574154c05ae60c113b41"
Sep 10 21:14:33.620577 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:33.620546    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:14:36.387368 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:36.387337    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"]
Sep 10 21:14:36.389570 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:36.389540    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-76b274-w7ffh"]
Sep 10 21:14:36.585327 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:36.585294    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="105d5a8c-745a-400b-ac95-f91bd2647737" path="/var/lib/kubelet/pods/105d5a8c-745a-400b-ac95-f91bd2647737/volumes"
Sep 10 21:14:39.361886 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:39.361848    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:14:40.631026 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:40.630997    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:14:55.206663 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.206625    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"]
Sep 10 21:14:55.207133 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.206904    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l" podUID="757237a7-e86c-4890-b909-ed7791d794d4" containerName="payload-processing" containerID="cri-o://628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c" gracePeriod=30
Sep 10 21:14:55.452499 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.452476    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:14:55.510509 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.510478    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/757237a7-e86c-4890-b909-ed7791d794d4-plugins-config-volume\") pod \"757237a7-e86c-4890-b909-ed7791d794d4\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") "
Sep 10 21:14:55.510650 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.510563    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlrbm\" (UniqueName: \"kubernetes.io/projected/757237a7-e86c-4890-b909-ed7791d794d4-kube-api-access-tlrbm\") pod \"757237a7-e86c-4890-b909-ed7791d794d4\" (UID: \"757237a7-e86c-4890-b909-ed7791d794d4\") "
Sep 10 21:14:55.510926 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.510901    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/757237a7-e86c-4890-b909-ed7791d794d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "757237a7-e86c-4890-b909-ed7791d794d4" (UID: "757237a7-e86c-4890-b909-ed7791d794d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:14:55.513025 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.512988    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/757237a7-e86c-4890-b909-ed7791d794d4-kube-api-access-tlrbm" (OuterVolumeSpecName: "kube-api-access-tlrbm") pod "757237a7-e86c-4890-b909-ed7791d794d4" (UID: "757237a7-e86c-4890-b909-ed7791d794d4"). InnerVolumeSpecName "kube-api-access-tlrbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:14:55.611932 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.611903    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/757237a7-e86c-4890-b909-ed7791d794d4-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.611932 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.611928    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlrbm\" (UniqueName: \"kubernetes.io/projected/757237a7-e86c-4890-b909-ed7791d794d4-kube-api-access-tlrbm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:14:55.772790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.772705    2557 generic.go:356] "Generic (PLEG): container finished" podID="757237a7-e86c-4890-b909-ed7791d794d4" containerID="628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c" exitCode=0
Sep 10 21:14:55.772790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.772756    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"
Sep 10 21:14:55.773000 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.772793    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l" event={"ID":"757237a7-e86c-4890-b909-ed7791d794d4","Type":"ContainerDied","Data":"628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c"}
Sep 10 21:14:55.773000 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.772847    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l" event={"ID":"757237a7-e86c-4890-b909-ed7791d794d4","Type":"ContainerDied","Data":"26a165092397e9ee4344929d418c81d3c33ceca2f8754312cd65b4fe80fe470a"}
Sep 10 21:14:55.773000 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.772869    2557 scope.go:117] "RemoveContainer" containerID="628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c"
Sep 10 21:14:55.782025 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.782009    2557 scope.go:117] "RemoveContainer" containerID="628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c"
Sep 10 21:14:55.782279 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:14:55.782263    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c\": container with ID starting with 628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c not found: ID does not exist" containerID="628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c"
Sep 10 21:14:55.782324 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.782287    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c"} err="failed to get container status \"628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c\": rpc error: code = NotFound desc = could not find container \"628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c\": container with ID starting with 628eee434715164f5b255a0a561a1f17bed55a1cb605d95358aea96ac599631c not found: ID does not exist"
Sep 10 21:14:55.797254 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.797228    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"]
Sep 10 21:14:55.799568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:55.799549    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-31e7cb-c7b9bf57-zv75l"]
Sep 10 21:14:56.586894 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:56.586863    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="757237a7-e86c-4890-b909-ed7791d794d4" path="/var/lib/kubelet/pods/757237a7-e86c-4890-b909-ed7791d794d4/volumes"
Sep 10 21:14:57.558236 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:14:57.558197    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:15:08.123284 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.123203    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"]
Sep 10 21:15:08.123701 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.123439    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt" podUID="508cb91d-f2ca-4fcb-8c13-943206a0e903" containerName="maas-api" containerID="cri-o://816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20" gracePeriod=30
Sep 10 21:15:08.185582 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.185547    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"]
Sep 10 21:15:08.185825 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.185780    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2" podUID="7a83608c-81f1-40eb-b2d9-8d3ce76c6597" containerName="payload-processing" containerID="cri-o://6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4" gracePeriod=30
Sep 10 21:15:08.444292 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.444266    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:15:08.446879 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.446861    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:15:08.515895 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.515865    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/508cb91d-f2ca-4fcb-8c13-943206a0e903-openshift-service-ca\") pod \"508cb91d-f2ca-4fcb-8c13-943206a0e903\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") "
Sep 10 21:15:08.515895 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.515895    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-maas-api-tls\") pod \"508cb91d-f2ca-4fcb-8c13-943206a0e903\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") "
Sep 10 21:15:08.516084 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.515934    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46hlc\" (UniqueName: \"kubernetes.io/projected/508cb91d-f2ca-4fcb-8c13-943206a0e903-kube-api-access-46hlc\") pod \"508cb91d-f2ca-4fcb-8c13-943206a0e903\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") "
Sep 10 21:15:08.516084 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.515987    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-metrics-certs\") pod \"508cb91d-f2ca-4fcb-8c13-943206a0e903\" (UID: \"508cb91d-f2ca-4fcb-8c13-943206a0e903\") "
Sep 10 21:15:08.516168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.516144    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2frqv\" (UniqueName: \"kubernetes.io/projected/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-kube-api-access-2frqv\") pod \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") "
Sep 10 21:15:08.516219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.516194    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-plugins-config-volume\") pod \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\" (UID: \"7a83608c-81f1-40eb-b2d9-8d3ce76c6597\") "
Sep 10 21:15:08.516273 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.516230    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508cb91d-f2ca-4fcb-8c13-943206a0e903-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "508cb91d-f2ca-4fcb-8c13-943206a0e903" (UID: "508cb91d-f2ca-4fcb-8c13-943206a0e903"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:08.516526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.516493    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/508cb91d-f2ca-4fcb-8c13-943206a0e903-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.516659 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.516635    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7a83608c-81f1-40eb-b2d9-8d3ce76c6597" (UID: "7a83608c-81f1-40eb-b2d9-8d3ce76c6597"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:15:08.518157 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.518127    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/508cb91d-f2ca-4fcb-8c13-943206a0e903-kube-api-access-46hlc" (OuterVolumeSpecName: "kube-api-access-46hlc") pod "508cb91d-f2ca-4fcb-8c13-943206a0e903" (UID: "508cb91d-f2ca-4fcb-8c13-943206a0e903"). InnerVolumeSpecName "kube-api-access-46hlc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:15:08.518269 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.518154    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "508cb91d-f2ca-4fcb-8c13-943206a0e903" (UID: "508cb91d-f2ca-4fcb-8c13-943206a0e903"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:15:08.518269 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.518154    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-kube-api-access-2frqv" (OuterVolumeSpecName: "kube-api-access-2frqv") pod "7a83608c-81f1-40eb-b2d9-8d3ce76c6597" (UID: "7a83608c-81f1-40eb-b2d9-8d3ce76c6597"). InnerVolumeSpecName "kube-api-access-2frqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:15:08.518269 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.518187    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "508cb91d-f2ca-4fcb-8c13-943206a0e903" (UID: "508cb91d-f2ca-4fcb-8c13-943206a0e903"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:15:08.617565 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.617540    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46hlc\" (UniqueName: \"kubernetes.io/projected/508cb91d-f2ca-4fcb-8c13-943206a0e903-kube-api-access-46hlc\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.617565 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.617562    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.617727 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.617576    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2frqv\" (UniqueName: \"kubernetes.io/projected/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-kube-api-access-2frqv\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.617727 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.617589    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a83608c-81f1-40eb-b2d9-8d3ce76c6597-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.617727 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.617601    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/508cb91d-f2ca-4fcb-8c13-943206a0e903-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:15:08.842299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.842264    2557 generic.go:356] "Generic (PLEG): container finished" podID="7a83608c-81f1-40eb-b2d9-8d3ce76c6597" containerID="6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4" exitCode=0
Sep 10 21:15:08.842451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.842312    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"
Sep 10 21:15:08.842451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.842349    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2" event={"ID":"7a83608c-81f1-40eb-b2d9-8d3ce76c6597","Type":"ContainerDied","Data":"6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4"}
Sep 10 21:15:08.842451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.842409    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2" event={"ID":"7a83608c-81f1-40eb-b2d9-8d3ce76c6597","Type":"ContainerDied","Data":"5987ef83fd8c1f2e02451806f91cd84949ac5479a9068e42a27e7013ab839766"}
Sep 10 21:15:08.842451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.842424    2557 scope.go:117] "RemoveContainer" containerID="6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4"
Sep 10 21:15:08.843906 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.843867    2557 generic.go:356] "Generic (PLEG): container finished" podID="508cb91d-f2ca-4fcb-8c13-943206a0e903" containerID="816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20" exitCode=0
Sep 10 21:15:08.843906 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.843901    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"
Sep 10 21:15:08.844043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.843991    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt" event={"ID":"508cb91d-f2ca-4fcb-8c13-943206a0e903","Type":"ContainerDied","Data":"816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20"}
Sep 10 21:15:08.844043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.844034    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt" event={"ID":"508cb91d-f2ca-4fcb-8c13-943206a0e903","Type":"ContainerDied","Data":"f8a92b822c46250f4e22e0fdd195cbf0d53bd8c6838fd8f9e9a7eec2f79a8eee"}
Sep 10 21:15:08.852158 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.852140    2557 scope.go:117] "RemoveContainer" containerID="6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4"
Sep 10 21:15:08.852440 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:15:08.852417    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4\": container with ID starting with 6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4 not found: ID does not exist" containerID="6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4"
Sep 10 21:15:08.852530 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.852449    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4"} err="failed to get container status \"6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4\": rpc error: code = NotFound desc = could not find container \"6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4\": container with ID starting with 6ab0923c6dad8e874577f698d43d9cfd5a9eb0d906c4d39c956b23968262f8f4 not found: ID does not exist"
Sep 10 21:15:08.852530 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.852470    2557 scope.go:117] "RemoveContainer" containerID="816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20"
Sep 10 21:15:08.862786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.862761    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"]
Sep 10 21:15:08.866195 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.866171    2557 scope.go:117] "RemoveContainer" containerID="816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20"
Sep 10 21:15:08.866631 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:15:08.866605    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20\": container with ID starting with 816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20 not found: ID does not exist" containerID="816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20"
Sep 10 21:15:08.866751 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.866638    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20"} err="failed to get container status \"816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20\": rpc error: code = NotFound desc = could not find container \"816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20\": container with ID starting with 816a1a0a7aa88201acb5caf850cdd00ecd0fd555f4cb1666360bac5702038a20 not found: ID does not exist"
Sep 10 21:15:08.867486 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.867469    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c6e2e41d-64bfc94697-4zkvt"]
Sep 10 21:15:08.878267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.878247    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"]
Sep 10 21:15:08.882648 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:08.882623    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c6e2e41d-55658b5585-jl2l2"]
Sep 10 21:15:10.585571 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:10.585540    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="508cb91d-f2ca-4fcb-8c13-943206a0e903" path="/var/lib/kubelet/pods/508cb91d-f2ca-4fcb-8c13-943206a0e903/volumes"
Sep 10 21:15:10.585953 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:10.585887    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a83608c-81f1-40eb-b2d9-8d3ce76c6597" path="/var/lib/kubelet/pods/7a83608c-81f1-40eb-b2d9-8d3ce76c6597/volumes"
Sep 10 21:15:11.540313 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:11.540280    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:15:12.150220 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150184    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"]
Sep 10 21:15:12.150568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150529    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="508cb91d-f2ca-4fcb-8c13-943206a0e903" containerName="maas-api"
Sep 10 21:15:12.150568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150539    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="508cb91d-f2ca-4fcb-8c13-943206a0e903" containerName="maas-api"
Sep 10 21:15:12.150568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150550    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="757237a7-e86c-4890-b909-ed7791d794d4" containerName="payload-processing"
Sep 10 21:15:12.150568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150556    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="757237a7-e86c-4890-b909-ed7791d794d4" containerName="payload-processing"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150573    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a83608c-81f1-40eb-b2d9-8d3ce76c6597" containerName="payload-processing"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150580    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a83608c-81f1-40eb-b2d9-8d3ce76c6597" containerName="payload-processing"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150591    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="105d5a8c-745a-400b-ac95-f91bd2647737" containerName="revoke-keys"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150596    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="105d5a8c-745a-400b-ac95-f91bd2647737" containerName="revoke-keys"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150647    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a83608c-81f1-40eb-b2d9-8d3ce76c6597" containerName="payload-processing"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150655    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="757237a7-e86c-4890-b909-ed7791d794d4" containerName="payload-processing"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150663    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="105d5a8c-745a-400b-ac95-f91bd2647737" containerName="revoke-keys"
Sep 10 21:15:12.150729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.150670    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="508cb91d-f2ca-4fcb-8c13-943206a0e903" containerName="maas-api"
Sep 10 21:15:12.153729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.153709    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.156321 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.156299    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-14acb0-dockercfg-cqmp9\""
Sep 10 21:15:12.156938 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.156914    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-14acb0\""
Sep 10 21:15:12.162545 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.162522    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"]
Sep 10 21:15:12.245026 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.244999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5438a58-1869-4579-8263-cb498e4be0bf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.245026 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.245029    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpnwm\" (UniqueName: \"kubernetes.io/projected/c5438a58-1869-4579-8263-cb498e4be0bf-kube-api-access-vpnwm\") pod \"payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.346468 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.346430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5438a58-1869-4579-8263-cb498e4be0bf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.346468 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.346467    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpnwm\" (UniqueName: \"kubernetes.io/projected/c5438a58-1869-4579-8263-cb498e4be0bf-kube-api-access-vpnwm\") pod \"payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.347130 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.347109    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5438a58-1869-4579-8263-cb498e4be0bf-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.369307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.369281    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpnwm\" (UniqueName: \"kubernetes.io/projected/c5438a58-1869-4579-8263-cb498e4be0bf-kube-api-access-vpnwm\") pod \"payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.464824 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.464797    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.736285 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.736258    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"]
Sep 10 21:15:12.737292 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:15:12.737269    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5438a58_1869_4579_8263_cb498e4be0bf.slice/crio-4bc2004e08277b84e52d3b6f576daf2cc5461ecdc1babb8daf7d7f4c97bf2cf5 WatchSource:0}: Error finding container 4bc2004e08277b84e52d3b6f576daf2cc5461ecdc1babb8daf7d7f4c97bf2cf5: Status 404 returned error can't find the container with id 4bc2004e08277b84e52d3b6f576daf2cc5461ecdc1babb8daf7d7f4c97bf2cf5
Sep 10 21:15:12.868529 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.868496    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q" event={"ID":"c5438a58-1869-4579-8263-cb498e4be0bf","Type":"ContainerStarted","Data":"8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2"}
Sep 10 21:15:12.868742 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.868552    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q" event={"ID":"c5438a58-1869-4579-8263-cb498e4be0bf","Type":"ContainerStarted","Data":"4bc2004e08277b84e52d3b6f576daf2cc5461ecdc1babb8daf7d7f4c97bf2cf5"}
Sep 10 21:15:12.868742 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.868637    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:12.890124 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:12.890077    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q" podStartSLOduration=0.890056406 podStartE2EDuration="890.056406ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:12.887634851 +0000 UTC m=+992.907040169" watchObservedRunningTime="2026-09-10 21:15:12.890056406 +0000 UTC m=+992.909461729"
Sep 10 21:15:22.140411 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:22.140380    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:15:23.876857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:23.876824    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:15:35.618963 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.618924    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"]
Sep 10 21:15:35.621607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.621589    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.624145 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.624127    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-290e03-dockercfg-kq2z7\""
Sep 10 21:15:35.624240 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.624127    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-290e03\""
Sep 10 21:15:35.633518 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.633492    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"]
Sep 10 21:15:35.728991 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.728955    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"]
Sep 10 21:15:35.731626 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.731604    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:35.745365 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.745342    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"]
Sep 10 21:15:35.753502 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.753480    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.753625 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.753589    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8hbm\" (UniqueName: \"kubernetes.io/projected/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-kube-api-access-k8hbm\") pod \"payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.854614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.854582    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d90316f7-9818-4ca4-bbf2-dc244ef132f5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:35.854787 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.854633    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf2gz\" (UniqueName: \"kubernetes.io/projected/d90316f7-9818-4ca4-bbf2-dc244ef132f5-kube-api-access-xf2gz\") pod \"payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:35.854787 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.854730    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.854787 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.854776    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8hbm\" (UniqueName: \"kubernetes.io/projected/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-kube-api-access-k8hbm\") pod \"payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.855644 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.855620    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.866071 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.866046    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8hbm\" (UniqueName: \"kubernetes.io/projected/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-kube-api-access-k8hbm\") pod \"payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.932782 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.931910    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:35.956174 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.956142    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d90316f7-9818-4ca4-bbf2-dc244ef132f5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:35.956432 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.956202    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf2gz\" (UniqueName: \"kubernetes.io/projected/d90316f7-9818-4ca4-bbf2-dc244ef132f5-kube-api-access-xf2gz\") pod \"payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:35.957064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.957041    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d90316f7-9818-4ca4-bbf2-dc244ef132f5-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:35.964885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:35.964860    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf2gz\" (UniqueName: \"kubernetes.io/projected/d90316f7-9818-4ca4-bbf2-dc244ef132f5-kube-api-access-xf2gz\") pod \"payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:36.043999 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:36.043967    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:36.212982 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:36.212953    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"]
Sep 10 21:15:36.214150 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:15:36.214122    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b0da8c_b897_4555_8d8c_8dfd46abf3a4.slice/crio-338c3fc2ad504725a8ebeadd1b4bc2da1c6df80bc01492da250ecf79684259dc WatchSource:0}: Error finding container 338c3fc2ad504725a8ebeadd1b4bc2da1c6df80bc01492da250ecf79684259dc: Status 404 returned error can't find the container with id 338c3fc2ad504725a8ebeadd1b4bc2da1c6df80bc01492da250ecf79684259dc
Sep 10 21:15:36.315514 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:36.315488    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"]
Sep 10 21:15:36.317168 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:15:36.317141    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90316f7_9818_4ca4_bbf2_dc244ef132f5.slice/crio-140f93caa66130ce97a69762b724e7821e5f83b5dbf1df248cc218c4d34091e9 WatchSource:0}: Error finding container 140f93caa66130ce97a69762b724e7821e5f83b5dbf1df248cc218c4d34091e9: Status 404 returned error can't find the container with id 140f93caa66130ce97a69762b724e7821e5f83b5dbf1df248cc218c4d34091e9
Sep 10 21:15:37.012561 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.012522    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p" event={"ID":"d90316f7-9818-4ca4-bbf2-dc244ef132f5","Type":"ContainerStarted","Data":"49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248"}
Sep 10 21:15:37.012959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.012590    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p" event={"ID":"d90316f7-9818-4ca4-bbf2-dc244ef132f5","Type":"ContainerStarted","Data":"140f93caa66130ce97a69762b724e7821e5f83b5dbf1df248cc218c4d34091e9"}
Sep 10 21:15:37.012959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.012620    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:37.014713 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.014667    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz" event={"ID":"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4","Type":"ContainerStarted","Data":"57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5"}
Sep 10 21:15:37.014831 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.014735    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz" event={"ID":"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4","Type":"ContainerStarted","Data":"338c3fc2ad504725a8ebeadd1b4bc2da1c6df80bc01492da250ecf79684259dc"}
Sep 10 21:15:37.014831 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.014779    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:37.030819 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.030770    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p" podStartSLOduration=2.030757825 podStartE2EDuration="2.030757825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:37.030224497 +0000 UTC m=+1017.049629815" watchObservedRunningTime="2026-09-10 21:15:37.030757825 +0000 UTC m=+1017.050163137"
Sep 10 21:15:37.050691 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:37.050636    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz" podStartSLOduration=2.050624035 podStartE2EDuration="2.050624035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:37.048229012 +0000 UTC m=+1017.067634333" watchObservedRunningTime="2026-09-10 21:15:37.050624035 +0000 UTC m=+1017.070029353"
Sep 10 21:15:39.439047 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:39.439011    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:15:40.922531 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:40.922459    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"]
Sep 10 21:15:40.927209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:40.927192    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:40.930235 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:40.930214    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-755926-dockercfg-7x99m\""
Sep 10 21:15:40.931196 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:40.931172    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-755926\""
Sep 10 21:15:40.939753 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:40.939733    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"]
Sep 10 21:15:40.997020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:40.996987    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz92h\" (UniqueName: \"kubernetes.io/projected/affb1540-b933-4758-b413-2435a249bb47-kube-api-access-hz92h\") pod \"payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:40.997178 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:40.997087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/affb1540-b933-4758-b413-2435a249bb47-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:41.098248 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:41.098214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz92h\" (UniqueName: \"kubernetes.io/projected/affb1540-b933-4758-b413-2435a249bb47-kube-api-access-hz92h\") pod \"payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:41.098422 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:41.098290    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/affb1540-b933-4758-b413-2435a249bb47-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:41.098925 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:41.098907    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/affb1540-b933-4758-b413-2435a249bb47-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:41.107602 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:41.107583    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz92h\" (UniqueName: \"kubernetes.io/projected/affb1540-b933-4758-b413-2435a249bb47-kube-api-access-hz92h\") pod \"payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:41.238325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:41.238293    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:41.513316 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:41.513284    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"]
Sep 10 21:15:41.514422 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:15:41.514395    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaffb1540_b933_4758_b413_2435a249bb47.slice/crio-1b6b66ab6862c91e7a23e1086702563e9a8f2d723d38427d2458eaf253a63d2b WatchSource:0}: Error finding container 1b6b66ab6862c91e7a23e1086702563e9a8f2d723d38427d2458eaf253a63d2b: Status 404 returned error can't find the container with id 1b6b66ab6862c91e7a23e1086702563e9a8f2d723d38427d2458eaf253a63d2b
Sep 10 21:15:42.043610 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:42.043579    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt" event={"ID":"affb1540-b933-4758-b413-2435a249bb47","Type":"ContainerStarted","Data":"c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe"}
Sep 10 21:15:42.044027 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:42.043635    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:15:42.044027 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:42.043652    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt" event={"ID":"affb1540-b933-4758-b413-2435a249bb47","Type":"ContainerStarted","Data":"1b6b66ab6862c91e7a23e1086702563e9a8f2d723d38427d2458eaf253a63d2b"}
Sep 10 21:15:42.062094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:42.062037    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt" podStartSLOduration=2.062017689 podStartE2EDuration="2.062017689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:15:42.060589215 +0000 UTC m=+1022.079994532" watchObservedRunningTime="2026-09-10 21:15:42.062017689 +0000 UTC m=+1022.081423009"
Sep 10 21:15:48.023195 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:48.023154    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:15:48.023656 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:48.023473    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:15:50.239242 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:50.239206    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:15:53.051293 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:15:53.051262    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:16:02.879931 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.879891    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"]
Sep 10 21:16:02.884420 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.884402    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:02.887567 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.887536    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-9f71ac-serving-cert\""
Sep 10 21:16:02.887702 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.887543    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:16:02.888427 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.888377    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:16:02.888522 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.888463    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:16:02.888643 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.888627    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:16:02.895536 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.895514    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"]
Sep 10 21:16:02.981564 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.981528    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:02.981564 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.981564    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql9rm\" (UniqueName: \"kubernetes.io/projected/530cc2d3-e831-4306-8c34-012b62324050-kube-api-access-ql9rm\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:02.981842 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.981601    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/530cc2d3-e831-4306-8c34-012b62324050-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:02.981842 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:02.981717    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.082270 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.082236    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.082270 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.082270    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql9rm\" (UniqueName: \"kubernetes.io/projected/530cc2d3-e831-4306-8c34-012b62324050-kube-api-access-ql9rm\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.082498 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.082315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/530cc2d3-e831-4306-8c34-012b62324050-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.082498 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:03.082390    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-serving-cert: secret "maas-api-e2e-shared-b-w2-9f71ac-serving-cert" not found
Sep 10 21:16:03.082498 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.082404    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.082498 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:03.082465    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls podName:530cc2d3-e831-4306-8c34-012b62324050 nodeName:}" failed. No retries permitted until 2026-09-10 21:16:03.582445191 +0000 UTC m=+1043.601850500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls") pod "maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2" (UID: "530cc2d3-e831-4306-8c34-012b62324050") : secret "maas-api-e2e-shared-b-w2-9f71ac-serving-cert" not found
Sep 10 21:16:03.083021 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.083001    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/530cc2d3-e831-4306-8c34-012b62324050-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.084796 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.084777    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.091340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.091314    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql9rm\" (UniqueName: \"kubernetes.io/projected/530cc2d3-e831-4306-8c34-012b62324050-kube-api-access-ql9rm\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.585468 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.585432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.587898 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.587867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:03.797414 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:03.797373    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:04.075548 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:04.075509    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"]
Sep 10 21:16:04.077278 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:04.077249    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530cc2d3_e831_4306_8c34_012b62324050.slice/crio-b46310c653335196bfe448dd58f2dc169f484c048de30acf6c195de727f6b18a WatchSource:0}: Error finding container b46310c653335196bfe448dd58f2dc169f484c048de30acf6c195de727f6b18a: Status 404 returned error can't find the container with id b46310c653335196bfe448dd58f2dc169f484c048de30acf6c195de727f6b18a
Sep 10 21:16:04.078651 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:04.078634    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:16:04.183313 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:04.183232    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2" event={"ID":"530cc2d3-e831-4306-8c34-012b62324050","Type":"ContainerStarted","Data":"b46310c653335196bfe448dd58f2dc169f484c048de30acf6c195de727f6b18a"}
Sep 10 21:16:05.190325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:05.190284    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2" event={"ID":"530cc2d3-e831-4306-8c34-012b62324050","Type":"ContainerStarted","Data":"63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d"}
Sep 10 21:16:05.190725 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:05.190354    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:05.208172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:05.208124    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2" podStartSLOduration=2.90822658 podStartE2EDuration="3.208108678s" totalImagesPullingTime="299.882098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:05.207359528 +0000 UTC m=+1045.226764847" watchObservedRunningTime="2026-09-10 21:16:05.208108678 +0000 UTC m=+1045.227514006"
Sep 10 21:16:08.937896 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:08.937858    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:16:09.542805 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.542770    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"]
Sep 10 21:16:09.550219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.549573    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.555392 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.555359    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"default-dockercfg-szwdh\""
Sep 10 21:16:09.555781 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.555758    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"openshift-service-ca.crt\""
Sep 10 21:16:09.556506 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.556488    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"kube-root-ca.crt\""
Sep 10 21:16:09.556587 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.556568    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"auth-test-model-14acb0-kserve-self-signed-certs\""
Sep 10 21:16:09.561823 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.561798    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"]
Sep 10 21:16:09.646289 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.646253    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrjgk\" (UniqueName: \"kubernetes.io/projected/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-kube-api-access-jrjgk\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.646462 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.646295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-model-cache\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.646462 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.646345    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-home\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.646462 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.646377    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tmp-dir\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.646462 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.646408    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-dshm\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.646672 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.646510    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tls-certs\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.747847 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.747820    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tls-certs\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.747989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.747880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrjgk\" (UniqueName: \"kubernetes.io/projected/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-kube-api-access-jrjgk\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.747989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.747902    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-model-cache\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.747989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.747922    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-home\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.747989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.747942    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tmp-dir\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.747989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.747962    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-dshm\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.748351 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.748325    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-home\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.748458 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.748367    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-model-cache\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.748458 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.748422    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tmp-dir\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.750113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.750092    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-dshm\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.750275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.750259    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tls-certs\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.757089 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.757068    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrjgk\" (UniqueName: \"kubernetes.io/projected/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-kube-api-access-jrjgk\") pod \"auth-test-model-14acb0-kserve-6cb686d6c-8qd4t\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:09.860446 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:09.860391    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:10.138341 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:10.137470    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"]
Sep 10 21:16:10.141671 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:10.141637    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2bc5619_e9d0_4137_aac3_bc94b1f4288f.slice/crio-864a279aee2b9127eea05142725b3e5a91f313830134ad8a2699b86c9ca4dd65 WatchSource:0}: Error finding container 864a279aee2b9127eea05142725b3e5a91f313830134ad8a2699b86c9ca4dd65: Status 404 returned error can't find the container with id 864a279aee2b9127eea05142725b3e5a91f313830134ad8a2699b86c9ca4dd65
Sep 10 21:16:10.220900 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:10.220865    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t" event={"ID":"a2bc5619-e9d0-4137-aac3-bc94b1f4288f","Type":"ContainerStarted","Data":"a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1"}
Sep 10 21:16:10.221000 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:10.220919    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t" event={"ID":"a2bc5619-e9d0-4137-aac3-bc94b1f4288f","Type":"ContainerStarted","Data":"864a279aee2b9127eea05142725b3e5a91f313830134ad8a2699b86c9ca4dd65"}
Sep 10 21:16:10.241029 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:10.240985    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t" podStartSLOduration=1.240969866 podStartE2EDuration="1.240969866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:10.240095612 +0000 UTC m=+1050.259500932" watchObservedRunningTime="2026-09-10 21:16:10.240969866 +0000 UTC m=+1050.260375184"
Sep 10 21:16:11.201820 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:11.201792    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:16:19.861381 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:19.861349    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:19.861831 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:19.861424    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:19.873978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:19.873956    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:20.293250 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:20.293215    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:16:22.332931 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:22.332895    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:16:26.855103 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:26.855068    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7632-36aa95"]
Sep 10 21:16:26.858813 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:26.858797    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:26.861336 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:26.861319    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-krpxc\""
Sep 10 21:16:26.866899 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:26.866878    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-36aa95"]
Sep 10 21:16:26.995304 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:26.995268    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnrrd\" (UniqueName: \"kubernetes.io/projected/4014a2bc-8ba0-4a5f-8b26-0ff16741325a-kube-api-access-lnrrd\") pod \"test-curl-7632-36aa95\" (UID: \"4014a2bc-8ba0-4a5f-8b26-0ff16741325a\") " pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:27.096100 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:27.096058    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrrd\" (UniqueName: \"kubernetes.io/projected/4014a2bc-8ba0-4a5f-8b26-0ff16741325a-kube-api-access-lnrrd\") pod \"test-curl-7632-36aa95\" (UID: \"4014a2bc-8ba0-4a5f-8b26-0ff16741325a\") " pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:27.104774 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:27.104741    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnrrd\" (UniqueName: \"kubernetes.io/projected/4014a2bc-8ba0-4a5f-8b26-0ff16741325a-kube-api-access-lnrrd\") pod \"test-curl-7632-36aa95\" (UID: \"4014a2bc-8ba0-4a5f-8b26-0ff16741325a\") " pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:27.169169 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:27.169092    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:27.442142 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:27.442118    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-36aa95"]
Sep 10 21:16:27.443921 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:27.443889    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4014a2bc_8ba0_4a5f_8b26_0ff16741325a.slice/crio-89d38e9493f80a32916369026dbb00e113779602d7a91781d449858bd2b5214f WatchSource:0}: Error finding container 89d38e9493f80a32916369026dbb00e113779602d7a91781d449858bd2b5214f: Status 404 returned error can't find the container with id 89d38e9493f80a32916369026dbb00e113779602d7a91781d449858bd2b5214f
Sep 10 21:16:28.325979 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.325945    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-36aa95" event={"ID":"4014a2bc-8ba0-4a5f-8b26-0ff16741325a","Type":"ContainerStarted","Data":"4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f"}
Sep 10 21:16:28.325979 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.325986    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-36aa95" event={"ID":"4014a2bc-8ba0-4a5f-8b26-0ff16741325a","Type":"ContainerStarted","Data":"89d38e9493f80a32916369026dbb00e113779602d7a91781d449858bd2b5214f"}
Sep 10 21:16:28.348999 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.348945    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7632-36aa95" podStartSLOduration=2.348929755 podStartE2EDuration="2.348929755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:28.346474119 +0000 UTC m=+1068.365879428" watchObservedRunningTime="2026-09-10 21:16:28.348929755 +0000 UTC m=+1068.368335073"
Sep 10 21:16:28.844144 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.844112    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"]
Sep 10 21:16:28.847630 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.847610    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:28.851276 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.851252    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"kube-root-ca.crt\""
Sep 10 21:16:28.851401 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.851260    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"auth-test-model-755926-kserve-self-signed-certs\""
Sep 10 21:16:28.851401 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.851313    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"default-dockercfg-t6cvs\""
Sep 10 21:16:28.851401 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.851353    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"openshift-service-ca.crt\""
Sep 10 21:16:28.856717 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.856695    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"]
Sep 10 21:16:28.911209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.911174    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-model-cache\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:28.911209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.911207    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-tmp-dir\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:28.911428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.911241    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c7c53f-b16e-442e-83c5-1dccb689ec05-tls-certs\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:28.911428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.911286    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-home\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:28.911428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.911317    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62z2w\" (UniqueName: \"kubernetes.io/projected/90c7c53f-b16e-442e-83c5-1dccb689ec05-kube-api-access-62z2w\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:28.911428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.911403    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-dshm\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:28.950396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.950370    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7632-36aa95"]
Sep 10 21:16:28.954319 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:28.954298    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7632-36aa95"]
Sep 10 21:16:29.012642 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.012569    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-home\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.012642 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.012616    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62z2w\" (UniqueName: \"kubernetes.io/projected/90c7c53f-b16e-442e-83c5-1dccb689ec05-kube-api-access-62z2w\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.012857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.012712    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-dshm\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.012857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.012782    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-model-cache\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.012857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.012812    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-tmp-dir\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.012857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.012837    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c7c53f-b16e-442e-83c5-1dccb689ec05-tls-certs\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.013063 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.013033    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-home\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.013117 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.013097    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-model-cache\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.013221 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.013185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-tmp-dir\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.014968 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.014945    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-dshm\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.015226 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.015198    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c7c53f-b16e-442e-83c5-1dccb689ec05-tls-certs\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.020990 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.020971    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62z2w\" (UniqueName: \"kubernetes.io/projected/90c7c53f-b16e-442e-83c5-1dccb689ec05-kube-api-access-62z2w\") pod \"auth-test-model-755926-kserve-87c44cddc-dbhq4\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") " pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.072154 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.072129    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7632-268d3f"]
Sep 10 21:16:29.072583 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.072568    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" containerName="test-curl-7632-36aa95"
Sep 10 21:16:29.072631 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.072585    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" containerName="test-curl-7632-36aa95"
Sep 10 21:16:29.072674 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.072665    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" containerName="test-curl-7632-36aa95"
Sep 10 21:16:29.075884 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.075869    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:29.080287 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.080267    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-268d3f"]
Sep 10 21:16:29.113617 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.113596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrrzr\" (UniqueName: \"kubernetes.io/projected/7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7-kube-api-access-rrrzr\") pod \"test-curl-7632-268d3f\" (UID: \"7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7\") " pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:29.157485 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.157464    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:29.215510 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.215107    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrrzr\" (UniqueName: \"kubernetes.io/projected/7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7-kube-api-access-rrrzr\") pod \"test-curl-7632-268d3f\" (UID: \"7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7\") " pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:29.225279 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.225255    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrrzr\" (UniqueName: \"kubernetes.io/projected/7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7-kube-api-access-rrrzr\") pod \"test-curl-7632-268d3f\" (UID: \"7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7\") " pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:29.332130 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:29.332053    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7632-36aa95" container="test-curl-7632-36aa95"
Sep 10 21:16:29.386400 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.386376    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:29.433421 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.433376    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"]
Sep 10 21:16:29.433703 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:29.433654    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90c7c53f_b16e_442e_83c5_1dccb689ec05.slice/crio-4087eeb95c09dea564398bbc2ed5b8d8c5b7c209495242de14a43dcaac0d976e WatchSource:0}: Error finding container 4087eeb95c09dea564398bbc2ed5b8d8c5b7c209495242de14a43dcaac0d976e: Status 404 returned error can't find the container with id 4087eeb95c09dea564398bbc2ed5b8d8c5b7c209495242de14a43dcaac0d976e
Sep 10 21:16:29.661430 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:29.661403    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-268d3f"]
Sep 10 21:16:29.662414 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:29.662391    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7696ea87_bec1_4c04_9d4f_3bf6dcd57ec7.slice/crio-3ac286017b2bd7b1829ae748b550082379a4aa8f2f911c48bb1ea14492287693 WatchSource:0}: Error finding container 3ac286017b2bd7b1829ae748b550082379a4aa8f2f911c48bb1ea14492287693: Status 404 returned error can't find the container with id 3ac286017b2bd7b1829ae748b550082379a4aa8f2f911c48bb1ea14492287693
Sep 10 21:16:30.338903 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.338862    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-268d3f" event={"ID":"7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7","Type":"ContainerStarted","Data":"a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c"}
Sep 10 21:16:30.338903 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.338903    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-268d3f" event={"ID":"7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7","Type":"ContainerStarted","Data":"3ac286017b2bd7b1829ae748b550082379a4aa8f2f911c48bb1ea14492287693"}
Sep 10 21:16:30.340631 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.340601    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4" event={"ID":"90c7c53f-b16e-442e-83c5-1dccb689ec05","Type":"ContainerStarted","Data":"b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92"}
Sep 10 21:16:30.340631 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.340635    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4" event={"ID":"90c7c53f-b16e-442e-83c5-1dccb689ec05","Type":"ContainerStarted","Data":"4087eeb95c09dea564398bbc2ed5b8d8c5b7c209495242de14a43dcaac0d976e"}
Sep 10 21:16:30.340858 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.340784    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7632-36aa95" podUID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" containerName="test-curl-7632-36aa95" containerID="cri-o://4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f" gracePeriod=2
Sep 10 21:16:30.356502 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.356420    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7632-268d3f" podStartSLOduration=1.356404314 podStartE2EDuration="1.356404314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:30.355841245 +0000 UTC m=+1070.375246563" watchObservedRunningTime="2026-09-10 21:16:30.356404314 +0000 UTC m=+1070.375809633"
Sep 10 21:16:30.374165 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.374124    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4" podStartSLOduration=2.374111776 podStartE2EDuration="2.374111776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:30.373182999 +0000 UTC m=+1070.392588318" watchObservedRunningTime="2026-09-10 21:16:30.374111776 +0000 UTC m=+1070.393517093"
Sep 10 21:16:30.375324 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:30.375301    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-36aa95\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:30.585179 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:30.585139    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-36aa95\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:30.946660 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.946629    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7632-268d3f"]
Sep 10 21:16:30.951316 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:30.951286    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7632-268d3f"]
Sep 10 21:16:31.069086 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.069014    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7632-261bf6"]
Sep 10 21:16:31.069416 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.069402    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" containerName="test-curl-7632-268d3f"
Sep 10 21:16:31.069416 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.069417    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" containerName="test-curl-7632-268d3f"
Sep 10 21:16:31.069498 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.069474    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" containerName="test-curl-7632-268d3f"
Sep 10 21:16:31.072526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.072511    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:31.078608 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.078582    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-261bf6"]
Sep 10 21:16:31.133234 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.133203    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4s9v\" (UniqueName: \"kubernetes.io/projected/5ed2efd5-173c-41ca-b744-1e250a1b8def-kube-api-access-c4s9v\") pod \"test-curl-7632-261bf6\" (UID: \"5ed2efd5-173c-41ca-b744-1e250a1b8def\") " pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:31.234562 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.234527    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4s9v\" (UniqueName: \"kubernetes.io/projected/5ed2efd5-173c-41ca-b744-1e250a1b8def-kube-api-access-c4s9v\") pod \"test-curl-7632-261bf6\" (UID: \"5ed2efd5-173c-41ca-b744-1e250a1b8def\") " pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:31.242587 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.242562    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4s9v\" (UniqueName: \"kubernetes.io/projected/5ed2efd5-173c-41ca-b744-1e250a1b8def-kube-api-access-c4s9v\") pod \"test-curl-7632-261bf6\" (UID: \"5ed2efd5-173c-41ca-b744-1e250a1b8def\") " pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:31.346311 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:31.346236    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7632-268d3f" container="test-curl-7632-268d3f"
Sep 10 21:16:31.382915 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.382891    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:31.651414 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:31.651392    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-261bf6"]
Sep 10 21:16:31.653269 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:31.653246    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ed2efd5_173c_41ca_b744_1e250a1b8def.slice/crio-da3a1c0ffbd725dd0fc54bf63b85146a99c558c02100a93298110065a1e07972 WatchSource:0}: Error finding container da3a1c0ffbd725dd0fc54bf63b85146a99c558c02100a93298110065a1e07972: Status 404 returned error can't find the container with id da3a1c0ffbd725dd0fc54bf63b85146a99c558c02100a93298110065a1e07972
Sep 10 21:16:32.353770 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.353737    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-261bf6" event={"ID":"5ed2efd5-173c-41ca-b744-1e250a1b8def","Type":"ContainerStarted","Data":"c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1"}
Sep 10 21:16:32.354164 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.353788    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-261bf6" event={"ID":"5ed2efd5-173c-41ca-b744-1e250a1b8def","Type":"ContainerStarted","Data":"da3a1c0ffbd725dd0fc54bf63b85146a99c558c02100a93298110065a1e07972"}
Sep 10 21:16:32.354164 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.353807    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7632-268d3f" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" containerName="test-curl-7632-268d3f" containerID="cri-o://a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c" gracePeriod=2
Sep 10 21:16:32.356457 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:32.356431    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-268d3f\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:32.373287 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.373242    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7632-261bf6" podStartSLOduration=1.373227213 podStartE2EDuration="1.373227213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:32.372107067 +0000 UTC m=+1072.391512386" watchObservedRunningTime="2026-09-10 21:16:32.373227213 +0000 UTC m=+1072.392632530"
Sep 10 21:16:32.374258 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:32.374232    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-268d3f\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:32.486368 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.486335    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:32.488613 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:32.488586    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-268d3f\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:32.547607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.547584    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnrrd\" (UniqueName: \"kubernetes.io/projected/4014a2bc-8ba0-4a5f-8b26-0ff16741325a-kube-api-access-lnrrd\") pod \"4014a2bc-8ba0-4a5f-8b26-0ff16741325a\" (UID: \"4014a2bc-8ba0-4a5f-8b26-0ff16741325a\") "
Sep 10 21:16:32.549600 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.549579    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4014a2bc-8ba0-4a5f-8b26-0ff16741325a-kube-api-access-lnrrd" (OuterVolumeSpecName: "kube-api-access-lnrrd") pod "4014a2bc-8ba0-4a5f-8b26-0ff16741325a" (UID: "4014a2bc-8ba0-4a5f-8b26-0ff16741325a"). InnerVolumeSpecName "kube-api-access-lnrrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:32.585503 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.585479    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" path="/var/lib/kubelet/pods/4014a2bc-8ba0-4a5f-8b26-0ff16741325a/volumes"
Sep 10 21:16:32.648788 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.648738    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnrrd\" (UniqueName: \"kubernetes.io/projected/4014a2bc-8ba0-4a5f-8b26-0ff16741325a-kube-api-access-lnrrd\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:16:32.843130 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.843066    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:16:32.956083 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.955987    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7632-261bf6"]
Sep 10 21:16:32.956330 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:32.956311    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7632-261bf6"]
Sep 10 21:16:33.060339 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.060300    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7632-f420d4"]
Sep 10 21:16:33.060811 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.060791    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed2efd5-173c-41ca-b744-1e250a1b8def" containerName="test-curl-7632-261bf6"
Sep 10 21:16:33.060811 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.060812    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed2efd5-173c-41ca-b744-1e250a1b8def" containerName="test-curl-7632-261bf6"
Sep 10 21:16:33.060979 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.060927    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed2efd5-173c-41ca-b744-1e250a1b8def" containerName="test-curl-7632-261bf6"
Sep 10 21:16:33.064267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.064246    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:33.069801 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.069776    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-f420d4"]
Sep 10 21:16:33.159024 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.158996    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vmhd\" (UniqueName: \"kubernetes.io/projected/6e8d2add-e16a-4a26-9b7c-7b9dd139f60e-kube-api-access-9vmhd\") pod \"test-curl-7632-f420d4\" (UID: \"6e8d2add-e16a-4a26-9b7c-7b9dd139f60e\") " pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:33.259840 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.259814    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vmhd\" (UniqueName: \"kubernetes.io/projected/6e8d2add-e16a-4a26-9b7c-7b9dd139f60e-kube-api-access-9vmhd\") pod \"test-curl-7632-f420d4\" (UID: \"6e8d2add-e16a-4a26-9b7c-7b9dd139f60e\") " pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:33.270014 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.269986    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vmhd\" (UniqueName: \"kubernetes.io/projected/6e8d2add-e16a-4a26-9b7c-7b9dd139f60e-kube-api-access-9vmhd\") pod \"test-curl-7632-f420d4\" (UID: \"6e8d2add-e16a-4a26-9b7c-7b9dd139f60e\") " pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:33.360987 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.360960    2557 generic.go:356] "Generic (PLEG): container finished" podID="4014a2bc-8ba0-4a5f-8b26-0ff16741325a" containerID="4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f" exitCode=137
Sep 10 21:16:33.361381 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.361009    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-36aa95"
Sep 10 21:16:33.361381 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.361039    2557 scope.go:117] "RemoveContainer" containerID="4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f"
Sep 10 21:16:33.361381 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:33.361271    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7632-261bf6" container="test-curl-7632-261bf6"
Sep 10 21:16:33.370102 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.370086    2557 scope.go:117] "RemoveContainer" containerID="4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f"
Sep 10 21:16:33.370359 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:33.370342    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f\": container with ID starting with 4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f not found: ID does not exist" containerID="4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f"
Sep 10 21:16:33.370396 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.370370    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f"} err="failed to get container status \"4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f\": rpc error: code = NotFound desc = could not find container \"4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f\": container with ID starting with 4c19871a1cee65a844dc4329e0ea5ffb9774c6cecfde0cd80977b659a946307f not found: ID does not exist"
Sep 10 21:16:33.375497 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.375481    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:33.646565 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:33.646529    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7632-f420d4"]
Sep 10 21:16:33.647671 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:33.647639    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e8d2add_e16a_4a26_9b7c_7b9dd139f60e.slice/crio-9c50ddabc21ff4466d3b270d9315b33c8ee24dba20108cabf8804fdd2c493cab WatchSource:0}: Error finding container 9c50ddabc21ff4466d3b270d9315b33c8ee24dba20108cabf8804fdd2c493cab: Status 404 returned error can't find the container with id 9c50ddabc21ff4466d3b270d9315b33c8ee24dba20108cabf8804fdd2c493cab
Sep 10 21:16:34.369858 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.369825    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-f420d4" event={"ID":"6e8d2add-e16a-4a26-9b7c-7b9dd139f60e","Type":"ContainerStarted","Data":"fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f"}
Sep 10 21:16:34.370160 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.369870    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7632-f420d4" event={"ID":"6e8d2add-e16a-4a26-9b7c-7b9dd139f60e","Type":"ContainerStarted","Data":"9c50ddabc21ff4466d3b270d9315b33c8ee24dba20108cabf8804fdd2c493cab"}
Sep 10 21:16:34.370160 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.369978    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7632-261bf6" podUID="5ed2efd5-173c-41ca-b744-1e250a1b8def" containerName="test-curl-7632-261bf6" containerID="cri-o://c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1" gracePeriod=2
Sep 10 21:16:34.386468 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.386393    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7632-f420d4" podStartSLOduration=1.386377625 podStartE2EDuration="1.386377625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:34.38549367 +0000 UTC m=+1074.404898990" watchObservedRunningTime="2026-09-10 21:16:34.386377625 +0000 UTC m=+1074.405782943"
Sep 10 21:16:34.387438 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:34.387414    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-261bf6\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="5ed2efd5-173c-41ca-b744-1e250a1b8def" pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:34.519259 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.519232    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:34.524616 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:34.524589    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-261bf6\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="5ed2efd5-173c-41ca-b744-1e250a1b8def" pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:34.572005 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.571978    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrrzr\" (UniqueName: \"kubernetes.io/projected/7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7-kube-api-access-rrrzr\") pod \"7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7\" (UID: \"7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7\") "
Sep 10 21:16:34.573932 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.573913    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7-kube-api-access-rrrzr" (OuterVolumeSpecName: "kube-api-access-rrrzr") pod "7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" (UID: "7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7"). InnerVolumeSpecName "kube-api-access-rrrzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:34.586045 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.586023    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" path="/var/lib/kubelet/pods/7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7/volumes"
Sep 10 21:16:34.673428 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:34.673375    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrrzr\" (UniqueName: \"kubernetes.io/projected/7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7-kube-api-access-rrrzr\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:16:35.021408 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:35.021380    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7632-f420d4"]
Sep 10 21:16:35.024054 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:35.024018    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7632-f420d4"]
Sep 10 21:16:35.376325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:35.376242    2557 generic.go:356] "Generic (PLEG): container finished" podID="7696ea87-bec1-4c04-9d4f-3bf6dcd57ec7" containerID="a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c" exitCode=137
Sep 10 21:16:35.376325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:35.376283    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-268d3f"
Sep 10 21:16:35.376776 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:35.376339    2557 scope.go:117] "RemoveContainer" containerID="a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c"
Sep 10 21:16:35.376776 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:35.376482    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7632-f420d4" container="test-curl-7632-f420d4"
Sep 10 21:16:35.387207 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:35.387188    2557 scope.go:117] "RemoveContainer" containerID="a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c"
Sep 10 21:16:35.387479 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:35.387460    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c\": container with ID starting with a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c not found: ID does not exist" containerID="a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c"
Sep 10 21:16:35.387542 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:35.387487    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c"} err="failed to get container status \"a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c\": rpc error: code = NotFound desc = could not find container \"a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c\": container with ID starting with a4bea33610aee6a3bfae6f9a1527084e651151c6b70a629b383fdc95acd6829c not found: ID does not exist"
Sep 10 21:16:36.382498 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:36.382467    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7632-f420d4" podUID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" containerName="test-curl-7632-f420d4" containerID="cri-o://fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f" gracePeriod=2
Sep 10 21:16:36.386272 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:36.386242    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-f420d4\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:36.554285 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:36.554253    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:36.556379 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:36.556346    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-f420d4\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:36.592328 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:36.592295    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4s9v\" (UniqueName: \"kubernetes.io/projected/5ed2efd5-173c-41ca-b744-1e250a1b8def-kube-api-access-c4s9v\") pod \"5ed2efd5-173c-41ca-b744-1e250a1b8def\" (UID: \"5ed2efd5-173c-41ca-b744-1e250a1b8def\") "
Sep 10 21:16:36.594295 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:36.594275    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed2efd5-173c-41ca-b744-1e250a1b8def-kube-api-access-c4s9v" (OuterVolumeSpecName: "kube-api-access-c4s9v") pod "5ed2efd5-173c-41ca-b744-1e250a1b8def" (UID: "5ed2efd5-173c-41ca-b744-1e250a1b8def"). InnerVolumeSpecName "kube-api-access-c4s9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:36.693998 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:36.693912    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4s9v\" (UniqueName: \"kubernetes.io/projected/5ed2efd5-173c-41ca-b744-1e250a1b8def-kube-api-access-c4s9v\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:16:37.388627 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:37.388592    2557 generic.go:356] "Generic (PLEG): container finished" podID="5ed2efd5-173c-41ca-b744-1e250a1b8def" containerID="c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1" exitCode=137
Sep 10 21:16:37.389070 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:37.388644    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-261bf6"
Sep 10 21:16:37.389070 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:37.388706    2557 scope.go:117] "RemoveContainer" containerID="c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1"
Sep 10 21:16:37.398549 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:37.398532    2557 scope.go:117] "RemoveContainer" containerID="c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1"
Sep 10 21:16:37.398836 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:37.398817    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1\": container with ID starting with c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1 not found: ID does not exist" containerID="c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1"
Sep 10 21:16:37.398887 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:37.398845    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1"} err="failed to get container status \"c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1\": rpc error: code = NotFound desc = could not find container \"c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1\": container with ID starting with c1a4b304caed0b050db191ef1bdff2eebcaba939a3f1f423c014645beeaf07f1 not found: ID does not exist"
Sep 10 21:16:38.534216 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:38.534193    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:38.586183 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:38.586154    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ed2efd5-173c-41ca-b744-1e250a1b8def" path="/var/lib/kubelet/pods/5ed2efd5-173c-41ca-b744-1e250a1b8def/volumes"
Sep 10 21:16:38.610238 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:38.610211    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vmhd\" (UniqueName: \"kubernetes.io/projected/6e8d2add-e16a-4a26-9b7c-7b9dd139f60e-kube-api-access-9vmhd\") pod \"6e8d2add-e16a-4a26-9b7c-7b9dd139f60e\" (UID: \"6e8d2add-e16a-4a26-9b7c-7b9dd139f60e\") "
Sep 10 21:16:38.612219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:38.612190    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e8d2add-e16a-4a26-9b7c-7b9dd139f60e-kube-api-access-9vmhd" (OuterVolumeSpecName: "kube-api-access-9vmhd") pod "6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" (UID: "6e8d2add-e16a-4a26-9b7c-7b9dd139f60e"). InnerVolumeSpecName "kube-api-access-9vmhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:16:38.711800 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:38.711721    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vmhd\" (UniqueName: \"kubernetes.io/projected/6e8d2add-e16a-4a26-9b7c-7b9dd139f60e-kube-api-access-9vmhd\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:16:39.158048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.158003    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:39.158239 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.158069    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:39.170882 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.170855    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:39.405471 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.405434    2557 generic.go:356] "Generic (PLEG): container finished" podID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" containerID="fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f" exitCode=137
Sep 10 21:16:39.405618 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.405476    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7632-f420d4"
Sep 10 21:16:39.405618 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.405531    2557 scope.go:117] "RemoveContainer" containerID="fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f"
Sep 10 21:16:39.415465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.415445    2557 scope.go:117] "RemoveContainer" containerID="fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f"
Sep 10 21:16:39.415746 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:16:39.415727    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f\": container with ID starting with fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f not found: ID does not exist" containerID="fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f"
Sep 10 21:16:39.415815 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.415758    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f"} err="failed to get container status \"fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f\": rpc error: code = NotFound desc = could not find container \"fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f\": container with ID starting with fff7a8e368b8c544145940e475f54c9e96d17267f7e834de363e52eea1cf557f not found: ID does not exist"
Sep 10 21:16:39.417860 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:39.417839    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:16:40.587162 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:40.587136    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" path="/var/lib/kubelet/pods/6e8d2add-e16a-4a26-9b7c-7b9dd139f60e/volumes"
Sep 10 21:16:41.278525 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.278483    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"]
Sep 10 21:16:41.279507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.279476    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" containerName="test-curl-7632-f420d4"
Sep 10 21:16:41.279507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.279505    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" containerName="test-curl-7632-f420d4"
Sep 10 21:16:41.279734 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.279720    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e8d2add-e16a-4a26-9b7c-7b9dd139f60e" containerName="test-curl-7632-f420d4"
Sep 10 21:16:41.289214 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.289188    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.290867 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.290839    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"]
Sep 10 21:16:41.292278 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.292253    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8b5fd067-dockercfg-djf5v\""
Sep 10 21:16:41.292278 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.292270    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8b5fd067\""
Sep 10 21:16:41.335452 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.335418    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6032a2b-b99c-47ad-8ad4-56630891aa3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.335589 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.335553    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck8f9\" (UniqueName: \"kubernetes.io/projected/a6032a2b-b99c-47ad-8ad4-56630891aa3a-kube-api-access-ck8f9\") pod \"payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.435997 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.435965    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck8f9\" (UniqueName: \"kubernetes.io/projected/a6032a2b-b99c-47ad-8ad4-56630891aa3a-kube-api-access-ck8f9\") pod \"payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.436162 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.436020    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6032a2b-b99c-47ad-8ad4-56630891aa3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.436662 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.436643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6032a2b-b99c-47ad-8ad4-56630891aa3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.444878 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.444854    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck8f9\" (UniqueName: \"kubernetes.io/projected/a6032a2b-b99c-47ad-8ad4-56630891aa3a-kube-api-access-ck8f9\") pod \"payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.601379 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.601300    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:41.879403 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:41.879373    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"]
Sep 10 21:16:41.881492 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:16:41.881464    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6032a2b_b99c_47ad_8ad4_56630891aa3a.slice/crio-74967d448372ce59fd95b83ee6c562719377a722c98694ce4b18b4a6b89a95ba WatchSource:0}: Error finding container 74967d448372ce59fd95b83ee6c562719377a722c98694ce4b18b4a6b89a95ba: Status 404 returned error can't find the container with id 74967d448372ce59fd95b83ee6c562719377a722c98694ce4b18b4a6b89a95ba
Sep 10 21:16:42.424355 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:42.424314    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx" event={"ID":"a6032a2b-b99c-47ad-8ad4-56630891aa3a","Type":"ContainerStarted","Data":"f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a"}
Sep 10 21:16:42.424543 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:42.424380    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx" event={"ID":"a6032a2b-b99c-47ad-8ad4-56630891aa3a","Type":"ContainerStarted","Data":"74967d448372ce59fd95b83ee6c562719377a722c98694ce4b18b4a6b89a95ba"}
Sep 10 21:16:42.424543 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:42.424501    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:16:42.440353 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:42.440301    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx" podStartSLOduration=1.440281119 podStartE2EDuration="1.440281119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:16:42.439746524 +0000 UTC m=+1082.459151842" watchObservedRunningTime="2026-09-10 21:16:42.440281119 +0000 UTC m=+1082.459686440"
Sep 10 21:16:51.534980 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:51.534941    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:16:53.431874 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:16:53.431839    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:17:04.495833 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:04.495798    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:17:09.126565 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.126530    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"]
Sep 10 21:17:09.130011 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.129988    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.133321 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.133296    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8b5fd067\"/\"kube-root-ca.crt\""
Sep 10 21:17:09.133321 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.133311    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8b5fd067\"/\"openshift-service-ca.crt\""
Sep 10 21:17:09.133528 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.133393    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8b5fd067\"/\"e2e-lifecycle-model-8b5fd067-kserve-self-signed-certs\""
Sep 10 21:17:09.133608 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.133589    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8b5fd067\"/\"default-dockercfg-gkgsr\""
Sep 10 21:17:09.137961 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.137873    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"]
Sep 10 21:17:09.273718 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.273648    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-home\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.273718 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.273719    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-dshm\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.273966 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.273743    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e47101-2875-4500-9417-69cd1f5fce4e-tls-certs\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.273966 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.273791    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-model-cache\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.273966 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.273854    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-tmp-dir\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.273966 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.273919    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-466lm\" (UniqueName: \"kubernetes.io/projected/a5e47101-2875-4500-9417-69cd1f5fce4e-kube-api-access-466lm\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.374995 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.374961    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-model-cache\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375019    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-tmp-dir\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375128    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-466lm\" (UniqueName: \"kubernetes.io/projected/a5e47101-2875-4500-9417-69cd1f5fce4e-kube-api-access-466lm\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375159    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-home\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375300 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375173    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-dshm\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375300 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375191    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e47101-2875-4500-9417-69cd1f5fce4e-tls-certs\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375425 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-model-cache\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375475 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375432    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-tmp-dir\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.375513 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.375469    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-home\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.377599 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.377537    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-dshm\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.377792 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.377775    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e47101-2875-4500-9417-69cd1f5fce4e-tls-certs\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.383171 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.383151    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-466lm\" (UniqueName: \"kubernetes.io/projected/a5e47101-2875-4500-9417-69cd1f5fce4e-kube-api-access-466lm\") pod \"e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") " pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.440845 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.440813    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:09.717386 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:09.717364    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"]
Sep 10 21:17:09.718948 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:17:09.718922    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5e47101_2875_4500_9417_69cd1f5fce4e.slice/crio-9bacb657a25cd9ad1a613dd02d81bc4f282f4e9cae79e025da532f21435b01b6 WatchSource:0}: Error finding container 9bacb657a25cd9ad1a613dd02d81bc4f282f4e9cae79e025da532f21435b01b6: Status 404 returned error can't find the container with id 9bacb657a25cd9ad1a613dd02d81bc4f282f4e9cae79e025da532f21435b01b6
Sep 10 21:17:10.592099 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:10.592057    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js" event={"ID":"a5e47101-2875-4500-9417-69cd1f5fce4e","Type":"ContainerStarted","Data":"5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e"}
Sep 10 21:17:10.592099 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:10.592104    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js" event={"ID":"a5e47101-2875-4500-9417-69cd1f5fce4e","Type":"ContainerStarted","Data":"9bacb657a25cd9ad1a613dd02d81bc4f282f4e9cae79e025da532f21435b01b6"}
Sep 10 21:17:10.615976 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:10.615928    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js" podStartSLOduration=1.615915637 podStartE2EDuration="1.615915637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:17:10.612287872 +0000 UTC m=+1110.631693189" watchObservedRunningTime="2026-09-10 21:17:10.615915637 +0000 UTC m=+1110.635320985"
Sep 10 21:17:19.441004 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:19.440972    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:19.441571 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:19.441026    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:19.453086 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:19.453064    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:19.654742 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:19.654710    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:17:21.634083 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:21.634043    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:17:39.542598 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:39.542567    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:17:48.835721 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:48.835669    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"]
Sep 10 21:17:48.836272 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:48.835978    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx" podUID="a6032a2b-b99c-47ad-8ad4-56630891aa3a" containerName="payload-pre-processing" containerID="cri-o://f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a" gracePeriod=30
Sep 10 21:17:49.088348 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.088289    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:17:49.214177 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.214147    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck8f9\" (UniqueName: \"kubernetes.io/projected/a6032a2b-b99c-47ad-8ad4-56630891aa3a-kube-api-access-ck8f9\") pod \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") "
Sep 10 21:17:49.214356 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.214227    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6032a2b-b99c-47ad-8ad4-56630891aa3a-plugins-config-volume\") pod \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\" (UID: \"a6032a2b-b99c-47ad-8ad4-56630891aa3a\") "
Sep 10 21:17:49.214625 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.214603    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6032a2b-b99c-47ad-8ad4-56630891aa3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a6032a2b-b99c-47ad-8ad4-56630891aa3a" (UID: "a6032a2b-b99c-47ad-8ad4-56630891aa3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:17:49.216196 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.216173    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6032a2b-b99c-47ad-8ad4-56630891aa3a-kube-api-access-ck8f9" (OuterVolumeSpecName: "kube-api-access-ck8f9") pod "a6032a2b-b99c-47ad-8ad4-56630891aa3a" (UID: "a6032a2b-b99c-47ad-8ad4-56630891aa3a"). InnerVolumeSpecName "kube-api-access-ck8f9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:17:49.315174 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.315143    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck8f9\" (UniqueName: \"kubernetes.io/projected/a6032a2b-b99c-47ad-8ad4-56630891aa3a-kube-api-access-ck8f9\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.315174 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.315171    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6032a2b-b99c-47ad-8ad4-56630891aa3a-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:17:49.824512 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.824477    2557 generic.go:356] "Generic (PLEG): container finished" podID="a6032a2b-b99c-47ad-8ad4-56630891aa3a" containerID="f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a" exitCode=0
Sep 10 21:17:49.824678 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.824522    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"
Sep 10 21:17:49.824678 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.824561    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx" event={"ID":"a6032a2b-b99c-47ad-8ad4-56630891aa3a","Type":"ContainerDied","Data":"f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a"}
Sep 10 21:17:49.824678 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.824617    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx" event={"ID":"a6032a2b-b99c-47ad-8ad4-56630891aa3a","Type":"ContainerDied","Data":"74967d448372ce59fd95b83ee6c562719377a722c98694ce4b18b4a6b89a95ba"}
Sep 10 21:17:49.824678 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.824642    2557 scope.go:117] "RemoveContainer" containerID="f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a"
Sep 10 21:17:49.834451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.834436    2557 scope.go:117] "RemoveContainer" containerID="f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a"
Sep 10 21:17:49.834744 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:17:49.834723    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a\": container with ID starting with f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a not found: ID does not exist" containerID="f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a"
Sep 10 21:17:49.834816 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.834751    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a"} err="failed to get container status \"f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a\": rpc error: code = NotFound desc = could not find container \"f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a\": container with ID starting with f03316791b49d72b1bbba340cd8b4b4e08a05390504d25c992f1a510c602070a not found: ID does not exist"
Sep 10 21:17:49.847858 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.847838    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"]
Sep 10 21:17:49.851757 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:49.851736    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-8b5fd067-59c9f747d-bjggx"]
Sep 10 21:17:50.587177 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:50.587138    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6032a2b-b99c-47ad-8ad4-56630891aa3a" path="/var/lib/kubelet/pods/a6032a2b-b99c-47ad-8ad4-56630891aa3a/volumes"
Sep 10 21:17:57.125250 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:57.125213    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"]
Sep 10 21:17:58.331745 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:17:58.331709    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:18:12.144731 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.144618    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js" podUID="a5e47101-2875-4500-9417-69cd1f5fce4e" containerName="main" containerID="cri-o://5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e" gracePeriod=45
Sep 10 21:18:12.401031 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.400964    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:18:12.425608 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.425578    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e47101-2875-4500-9417-69cd1f5fce4e-tls-certs\") pod \"a5e47101-2875-4500-9417-69cd1f5fce4e\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") "
Sep 10 21:18:12.425807 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.425650    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-dshm\") pod \"a5e47101-2875-4500-9417-69cd1f5fce4e\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") "
Sep 10 21:18:12.425807 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.425698    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-466lm\" (UniqueName: \"kubernetes.io/projected/a5e47101-2875-4500-9417-69cd1f5fce4e-kube-api-access-466lm\") pod \"a5e47101-2875-4500-9417-69cd1f5fce4e\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") "
Sep 10 21:18:12.425807 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.425727    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-model-cache\") pod \"a5e47101-2875-4500-9417-69cd1f5fce4e\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") "
Sep 10 21:18:12.425807 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.425781    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-home\") pod \"a5e47101-2875-4500-9417-69cd1f5fce4e\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") "
Sep 10 21:18:12.425807 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.425802    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-tmp-dir\") pod \"a5e47101-2875-4500-9417-69cd1f5fce4e\" (UID: \"a5e47101-2875-4500-9417-69cd1f5fce4e\") "
Sep 10 21:18:12.426357 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.426317    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5e47101-2875-4500-9417-69cd1f5fce4e" (UID: "a5e47101-2875-4500-9417-69cd1f5fce4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:12.426509 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.426416    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-home" (OuterVolumeSpecName: "home") pod "a5e47101-2875-4500-9417-69cd1f5fce4e" (UID: "a5e47101-2875-4500-9417-69cd1f5fce4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:12.426639 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.426618    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5e47101-2875-4500-9417-69cd1f5fce4e" (UID: "a5e47101-2875-4500-9417-69cd1f5fce4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:12.428571 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.428531    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5e47101-2875-4500-9417-69cd1f5fce4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5e47101-2875-4500-9417-69cd1f5fce4e" (UID: "a5e47101-2875-4500-9417-69cd1f5fce4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:18:12.428571 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.428533    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e47101-2875-4500-9417-69cd1f5fce4e-kube-api-access-466lm" (OuterVolumeSpecName: "kube-api-access-466lm") pod "a5e47101-2875-4500-9417-69cd1f5fce4e" (UID: "a5e47101-2875-4500-9417-69cd1f5fce4e"). InnerVolumeSpecName "kube-api-access-466lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:12.428985 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.428769    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-dshm" (OuterVolumeSpecName: "dshm") pod "a5e47101-2875-4500-9417-69cd1f5fce4e" (UID: "a5e47101-2875-4500-9417-69cd1f5fce4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:18:12.526609 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.526575    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5e47101-2875-4500-9417-69cd1f5fce4e-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:18:12.526609 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.526604    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:18:12.526609 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.526613    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-466lm\" (UniqueName: \"kubernetes.io/projected/a5e47101-2875-4500-9417-69cd1f5fce4e-kube-api-access-466lm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:18:12.526868 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.526622    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:18:12.526868 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.526630    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:18:12.526868 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.526638    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5e47101-2875-4500-9417-69cd1f5fce4e-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:18:12.952673 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.952633    2557 generic.go:356] "Generic (PLEG): container finished" podID="a5e47101-2875-4500-9417-69cd1f5fce4e" containerID="5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e" exitCode=0
Sep 10 21:18:12.952859 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.952678    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"
Sep 10 21:18:12.952859 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.952711    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js" event={"ID":"a5e47101-2875-4500-9417-69cd1f5fce4e","Type":"ContainerDied","Data":"5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e"}
Sep 10 21:18:12.952859 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.952760    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js" event={"ID":"a5e47101-2875-4500-9417-69cd1f5fce4e","Type":"ContainerDied","Data":"9bacb657a25cd9ad1a613dd02d81bc4f282f4e9cae79e025da532f21435b01b6"}
Sep 10 21:18:12.952859 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.952778    2557 scope.go:117] "RemoveContainer" containerID="5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e"
Sep 10 21:18:12.964361 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.964288    2557 scope.go:117] "RemoveContainer" containerID="5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e"
Sep 10 21:18:12.964781 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:18:12.964760    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e\": container with ID starting with 5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e not found: ID does not exist" containerID="5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e"
Sep 10 21:18:12.964864 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.964791    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e"} err="failed to get container status \"5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e\": rpc error: code = NotFound desc = could not find container \"5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e\": container with ID starting with 5e788424f60b134c84119dfa2856ae35930ec1dfbc44fe6dca9f6f138f055d9e not found: ID does not exist"
Sep 10 21:18:12.969307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.969284    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"]
Sep 10 21:18:12.974062 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:12.974040    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8b5fd067/e2e-lifecycle-model-8b5fd067-kserve-6ff4dff655-p55js"]
Sep 10 21:18:14.586204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:14.586166    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5e47101-2875-4500-9417-69cd1f5fce4e" path="/var/lib/kubelet/pods/a5e47101-2875-4500-9417-69cd1f5fce4e/volumes"
Sep 10 21:18:14.737199 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:14.731564    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:18:29.644267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:29.644229    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:18:38.715258 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.715216    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7635-105242"]
Sep 10 21:18:38.715827 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.715808    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6032a2b-b99c-47ad-8ad4-56630891aa3a" containerName="payload-pre-processing"
Sep 10 21:18:38.715940 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.715829    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6032a2b-b99c-47ad-8ad4-56630891aa3a" containerName="payload-pre-processing"
Sep 10 21:18:38.715940 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.715844    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e47101-2875-4500-9417-69cd1f5fce4e" containerName="main"
Sep 10 21:18:38.715940 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.715852    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e47101-2875-4500-9417-69cd1f5fce4e" containerName="main"
Sep 10 21:18:38.716102 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.715943    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5e47101-2875-4500-9417-69cd1f5fce4e" containerName="main"
Sep 10 21:18:38.716102 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.715958    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6032a2b-b99c-47ad-8ad4-56630891aa3a" containerName="payload-pre-processing"
Sep 10 21:18:38.720471 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.720451    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:38.723184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.723160    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-krpxc\""
Sep 10 21:18:38.726282 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.726260    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7635-105242"]
Sep 10 21:18:38.852481 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.852450    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nwgj\" (UniqueName: \"kubernetes.io/projected/120c2324-24b2-42e7-93b5-53064001ac3e-kube-api-access-8nwgj\") pod \"test-curl-7635-105242\" (UID: \"120c2324-24b2-42e7-93b5-53064001ac3e\") " pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:38.953178 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.953148    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nwgj\" (UniqueName: \"kubernetes.io/projected/120c2324-24b2-42e7-93b5-53064001ac3e-kube-api-access-8nwgj\") pod \"test-curl-7635-105242\" (UID: \"120c2324-24b2-42e7-93b5-53064001ac3e\") " pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:38.962948 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:38.962927    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nwgj\" (UniqueName: \"kubernetes.io/projected/120c2324-24b2-42e7-93b5-53064001ac3e-kube-api-access-8nwgj\") pod \"test-curl-7635-105242\" (UID: \"120c2324-24b2-42e7-93b5-53064001ac3e\") " pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:39.032606 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:39.032538    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:39.304382 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:39.304357    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7635-105242"]
Sep 10 21:18:39.306020 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:18:39.305982    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod120c2324_24b2_42e7_93b5_53064001ac3e.slice/crio-0fed0ae07106a7488477be7d544154e0d531925c1c98b68cafff7288f86a41a1 WatchSource:0}: Error finding container 0fed0ae07106a7488477be7d544154e0d531925c1c98b68cafff7288f86a41a1: Status 404 returned error can't find the container with id 0fed0ae07106a7488477be7d544154e0d531925c1c98b68cafff7288f86a41a1
Sep 10 21:18:40.108355 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:40.108319    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7635-105242" event={"ID":"120c2324-24b2-42e7-93b5-53064001ac3e","Type":"ContainerStarted","Data":"819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263"}
Sep 10 21:18:40.108355 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:40.108360    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7635-105242" event={"ID":"120c2324-24b2-42e7-93b5-53064001ac3e","Type":"ContainerStarted","Data":"0fed0ae07106a7488477be7d544154e0d531925c1c98b68cafff7288f86a41a1"}
Sep 10 21:18:40.126528 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:40.126478    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7635-105242" podStartSLOduration=2.126461862 podStartE2EDuration="2.126461862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:18:40.124960631 +0000 UTC m=+1200.144365948" watchObservedRunningTime="2026-09-10 21:18:40.126461862 +0000 UTC m=+1200.145867180"
Sep 10 21:18:40.475190 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:40.475160    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:18:40.479128 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:40.479104    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:18:40.756082 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:40.756003    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7635-105242"]
Sep 10 21:18:40.763404 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:40.763229    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7635-105242"]
Sep 10 21:18:41.113657 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:18:41.113584    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7635-105242" container="test-curl-7635-105242"
Sep 10 21:18:42.118562 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:42.118521    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7635-105242" podUID="120c2324-24b2-42e7-93b5-53064001ac3e" containerName="test-curl-7635-105242" containerID="cri-o://819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263" gracePeriod=2
Sep 10 21:18:42.120727 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:18:42.120694    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7635-105242\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="120c2324-24b2-42e7-93b5-53064001ac3e" pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:44.293742 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:44.293715    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:44.399072 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:44.398994    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nwgj\" (UniqueName: \"kubernetes.io/projected/120c2324-24b2-42e7-93b5-53064001ac3e-kube-api-access-8nwgj\") pod \"120c2324-24b2-42e7-93b5-53064001ac3e\" (UID: \"120c2324-24b2-42e7-93b5-53064001ac3e\") "
Sep 10 21:18:44.401072 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:44.401032    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120c2324-24b2-42e7-93b5-53064001ac3e-kube-api-access-8nwgj" (OuterVolumeSpecName: "kube-api-access-8nwgj") pod "120c2324-24b2-42e7-93b5-53064001ac3e" (UID: "120c2324-24b2-42e7-93b5-53064001ac3e"). InnerVolumeSpecName "kube-api-access-8nwgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:18:44.500729 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:44.500701    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nwgj\" (UniqueName: \"kubernetes.io/projected/120c2324-24b2-42e7-93b5-53064001ac3e-kube-api-access-8nwgj\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:18:44.585830 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:44.585802    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="120c2324-24b2-42e7-93b5-53064001ac3e" path="/var/lib/kubelet/pods/120c2324-24b2-42e7-93b5-53064001ac3e/volumes"
Sep 10 21:18:45.137191 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:45.137152    2557 generic.go:356] "Generic (PLEG): container finished" podID="120c2324-24b2-42e7-93b5-53064001ac3e" containerID="819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263" exitCode=137
Sep 10 21:18:45.137352 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:45.137202    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-105242"
Sep 10 21:18:45.137352 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:45.137246    2557 scope.go:117] "RemoveContainer" containerID="819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263"
Sep 10 21:18:45.146118 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:45.146101    2557 scope.go:117] "RemoveContainer" containerID="819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263"
Sep 10 21:18:45.146407 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:18:45.146384    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263\": container with ID starting with 819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263 not found: ID does not exist" containerID="819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263"
Sep 10 21:18:45.146502 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:45.146414    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263"} err="failed to get container status \"819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263\": rpc error: code = NotFound desc = could not find container \"819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263\": container with ID starting with 819bcdccd3ab8022a9842a4792c02e1a25514d3cdbc22725fea3564836d97263 not found: ID does not exist"
Sep 10 21:18:46.547341 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:46.547307    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:18:46.798813 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:46.798733    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"]
Sep 10 21:18:48.842544 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:18:48.842501    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"]
Sep 10 21:19:01.817767 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:01.817725    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t" podUID="a2bc5619-e9d0-4137-aac3-bc94b1f4288f" containerName="main" containerID="cri-o://a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1" gracePeriod=45
Sep 10 21:19:02.070557 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.070499    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:19:02.171857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.171826    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tmp-dir\") pod \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") "
Sep 10 21:19:02.171857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.171860    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-home\") pod \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") "
Sep 10 21:19:02.172110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.171890    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tls-certs\") pod \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") "
Sep 10 21:19:02.172110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.171928    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-model-cache\") pod \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") "
Sep 10 21:19:02.172110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.171964    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrjgk\" (UniqueName: \"kubernetes.io/projected/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-kube-api-access-jrjgk\") pod \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") "
Sep 10 21:19:02.172110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.171991    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-dshm\") pod \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\" (UID: \"a2bc5619-e9d0-4137-aac3-bc94b1f4288f\") "
Sep 10 21:19:02.172331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.172143    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-home" (OuterVolumeSpecName: "home") pod "a2bc5619-e9d0-4137-aac3-bc94b1f4288f" (UID: "a2bc5619-e9d0-4137-aac3-bc94b1f4288f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:02.172331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.172177    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2bc5619-e9d0-4137-aac3-bc94b1f4288f" (UID: "a2bc5619-e9d0-4137-aac3-bc94b1f4288f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:02.172331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.172234    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2bc5619-e9d0-4137-aac3-bc94b1f4288f" (UID: "a2bc5619-e9d0-4137-aac3-bc94b1f4288f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:02.172331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.172307    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:02.172331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.172326    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:02.172551 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.172337    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:02.174214 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.174183    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2bc5619-e9d0-4137-aac3-bc94b1f4288f" (UID: "a2bc5619-e9d0-4137-aac3-bc94b1f4288f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:19:02.174327 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.174223    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-dshm" (OuterVolumeSpecName: "dshm") pod "a2bc5619-e9d0-4137-aac3-bc94b1f4288f" (UID: "a2bc5619-e9d0-4137-aac3-bc94b1f4288f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:02.174327 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.174240    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-kube-api-access-jrjgk" (OuterVolumeSpecName: "kube-api-access-jrjgk") pod "a2bc5619-e9d0-4137-aac3-bc94b1f4288f" (UID: "a2bc5619-e9d0-4137-aac3-bc94b1f4288f"). InnerVolumeSpecName "kube-api-access-jrjgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:19:02.245954 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.245925    2557 generic.go:356] "Generic (PLEG): container finished" podID="a2bc5619-e9d0-4137-aac3-bc94b1f4288f" containerID="a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1" exitCode=0
Sep 10 21:19:02.246087 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.245967    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"
Sep 10 21:19:02.246087 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.246013    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t" event={"ID":"a2bc5619-e9d0-4137-aac3-bc94b1f4288f","Type":"ContainerDied","Data":"a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1"}
Sep 10 21:19:02.246087 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.246058    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t" event={"ID":"a2bc5619-e9d0-4137-aac3-bc94b1f4288f","Type":"ContainerDied","Data":"864a279aee2b9127eea05142725b3e5a91f313830134ad8a2699b86c9ca4dd65"}
Sep 10 21:19:02.246087 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.246080    2557 scope.go:117] "RemoveContainer" containerID="a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1"
Sep 10 21:19:02.255795 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.255773    2557 scope.go:117] "RemoveContainer" containerID="a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1"
Sep 10 21:19:02.256041 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:19:02.256024    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1\": container with ID starting with a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1 not found: ID does not exist" containerID="a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1"
Sep 10 21:19:02.256085 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.256047    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1"} err="failed to get container status \"a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1\": rpc error: code = NotFound desc = could not find container \"a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1\": container with ID starting with a3c842c723dae47016bd36d5ac159bda4bfdcea18de3f04653aba9f9d7f40df1 not found: ID does not exist"
Sep 10 21:19:02.267814 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.267793    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"]
Sep 10 21:19:02.271349 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.271328    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/auth-test-model-14acb0-kserve-6cb686d6c-8qd4t"]
Sep 10 21:19:02.273231 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.273213    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:02.273289 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.273234    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:02.273289 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.273244    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrjgk\" (UniqueName: \"kubernetes.io/projected/a2bc5619-e9d0-4137-aac3-bc94b1f4288f-kube-api-access-jrjgk\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:02.585900 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.585869    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2bc5619-e9d0-4137-aac3-bc94b1f4288f" path="/var/lib/kubelet/pods/a2bc5619-e9d0-4137-aac3-bc94b1f4288f/volumes"
Sep 10 21:19:02.740524 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.740487    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:19:02.744577 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.744541    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"]
Sep 10 21:19:02.744959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.744944    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2bc5619-e9d0-4137-aac3-bc94b1f4288f" containerName="main"
Sep 10 21:19:02.744959 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.744960    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2bc5619-e9d0-4137-aac3-bc94b1f4288f" containerName="main"
Sep 10 21:19:02.745081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.744967    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120c2324-24b2-42e7-93b5-53064001ac3e" containerName="test-curl-7635-105242"
Sep 10 21:19:02.745081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.744973    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="120c2324-24b2-42e7-93b5-53064001ac3e" containerName="test-curl-7635-105242"
Sep 10 21:19:02.745081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.745049    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2bc5619-e9d0-4137-aac3-bc94b1f4288f" containerName="main"
Sep 10 21:19:02.745081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.745057    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="120c2324-24b2-42e7-93b5-53064001ac3e" containerName="test-curl-7635-105242"
Sep 10 21:19:02.749163 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.749145    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.752297 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.752276    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"kube-root-ca.crt\""
Sep 10 21:19:02.752417 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.752293    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"openshift-service-ca.crt\""
Sep 10 21:19:02.752417 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.752296    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"default-dockercfg-szwdh\""
Sep 10 21:19:02.752417 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.752276    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"sub-test-model-14acb0-kserve-self-signed-certs\""
Sep 10 21:19:02.759798 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.759777    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"]
Sep 10 21:19:02.779092 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.779068    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-dshm\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.779193 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.779101    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tmp-dir\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.779193 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.779129    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5zrj\" (UniqueName: \"kubernetes.io/projected/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-kube-api-access-f5zrj\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.779325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.779224    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-model-cache\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.779325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.779289    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tls-certs\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.779431 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.779398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-home\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880181 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880121    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5zrj\" (UniqueName: \"kubernetes.io/projected/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-kube-api-access-f5zrj\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880181 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880158    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-model-cache\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tls-certs\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880231    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-home\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880261    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-dshm\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880288    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tmp-dir\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880595    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-model-cache\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880631    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tmp-dir\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.880895 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.880667    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-home\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.882547 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.882525    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-dshm\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.882855 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.882831    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tls-certs\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:02.889190 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:02.889172    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5zrj\" (UniqueName: \"kubernetes.io/projected/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-kube-api-access-f5zrj\") pod \"sub-test-model-14acb0-kserve-7cfcf6cf77-srqds\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:03.059130 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:03.059100    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:03.337781 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:03.337752    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"]
Sep 10 21:19:03.339221 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:19:03.339179    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd5bd384_4cd2_4a9d_b9e6_3f6295a8e3ef.slice/crio-ae23c05df656c0afc9e3ba68b4846e929b70b4eed80014a341ff988ce71b8dfc WatchSource:0}: Error finding container ae23c05df656c0afc9e3ba68b4846e929b70b4eed80014a341ff988ce71b8dfc: Status 404 returned error can't find the container with id ae23c05df656c0afc9e3ba68b4846e929b70b4eed80014a341ff988ce71b8dfc
Sep 10 21:19:03.863016 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:03.862976    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4" podUID="90c7c53f-b16e-442e-83c5-1dccb689ec05" containerName="main" containerID="cri-o://b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92" gracePeriod=45
Sep 10 21:19:03.903280 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:19:03.903250    2557 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90c7c53f_b16e_442e_83c5_1dccb689ec05.slice/crio-b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90c7c53f_b16e_442e_83c5_1dccb689ec05.slice/crio-conmon-b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:19:04.110855 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.110827    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:19:04.190639 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190574    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-dshm\") pod \"90c7c53f-b16e-442e-83c5-1dccb689ec05\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") "
Sep 10 21:19:04.190639 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190609    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-model-cache\") pod \"90c7c53f-b16e-442e-83c5-1dccb689ec05\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") "
Sep 10 21:19:04.190639 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190626    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-home\") pod \"90c7c53f-b16e-442e-83c5-1dccb689ec05\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") "
Sep 10 21:19:04.190912 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190655    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-tmp-dir\") pod \"90c7c53f-b16e-442e-83c5-1dccb689ec05\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") "
Sep 10 21:19:04.190912 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190732    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c7c53f-b16e-442e-83c5-1dccb689ec05-tls-certs\") pod \"90c7c53f-b16e-442e-83c5-1dccb689ec05\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") "
Sep 10 21:19:04.190912 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190767    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62z2w\" (UniqueName: \"kubernetes.io/projected/90c7c53f-b16e-442e-83c5-1dccb689ec05-kube-api-access-62z2w\") pod \"90c7c53f-b16e-442e-83c5-1dccb689ec05\" (UID: \"90c7c53f-b16e-442e-83c5-1dccb689ec05\") "
Sep 10 21:19:04.191074 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190914    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-model-cache" (OuterVolumeSpecName: "model-cache") pod "90c7c53f-b16e-442e-83c5-1dccb689ec05" (UID: "90c7c53f-b16e-442e-83c5-1dccb689ec05"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:04.191074 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190898    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-home" (OuterVolumeSpecName: "home") pod "90c7c53f-b16e-442e-83c5-1dccb689ec05" (UID: "90c7c53f-b16e-442e-83c5-1dccb689ec05"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:04.191074 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.190942    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "90c7c53f-b16e-442e-83c5-1dccb689ec05" (UID: "90c7c53f-b16e-442e-83c5-1dccb689ec05"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:04.191074 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.191067    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:04.191261 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.191087    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:04.191261 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.191100    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:04.192927 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.192899    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-dshm" (OuterVolumeSpecName: "dshm") pod "90c7c53f-b16e-442e-83c5-1dccb689ec05" (UID: "90c7c53f-b16e-442e-83c5-1dccb689ec05"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:19:04.193029 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.192983    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90c7c53f-b16e-442e-83c5-1dccb689ec05-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "90c7c53f-b16e-442e-83c5-1dccb689ec05" (UID: "90c7c53f-b16e-442e-83c5-1dccb689ec05"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:19:04.193029 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.192982    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90c7c53f-b16e-442e-83c5-1dccb689ec05-kube-api-access-62z2w" (OuterVolumeSpecName: "kube-api-access-62z2w") pod "90c7c53f-b16e-442e-83c5-1dccb689ec05" (UID: "90c7c53f-b16e-442e-83c5-1dccb689ec05"). InnerVolumeSpecName "kube-api-access-62z2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:19:04.259626 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.259599    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds" event={"ID":"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef","Type":"ContainerStarted","Data":"8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0"}
Sep 10 21:19:04.259777 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.259639    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds" event={"ID":"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef","Type":"ContainerStarted","Data":"ae23c05df656c0afc9e3ba68b4846e929b70b4eed80014a341ff988ce71b8dfc"}
Sep 10 21:19:04.261010 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.260969    2557 generic.go:356] "Generic (PLEG): container finished" podID="90c7c53f-b16e-442e-83c5-1dccb689ec05" containerID="b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92" exitCode=0
Sep 10 21:19:04.261010 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.261005    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"
Sep 10 21:19:04.261161 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.261018    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4" event={"ID":"90c7c53f-b16e-442e-83c5-1dccb689ec05","Type":"ContainerDied","Data":"b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92"}
Sep 10 21:19:04.261161 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.261049    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4" event={"ID":"90c7c53f-b16e-442e-83c5-1dccb689ec05","Type":"ContainerDied","Data":"4087eeb95c09dea564398bbc2ed5b8d8c5b7c209495242de14a43dcaac0d976e"}
Sep 10 21:19:04.261161 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.261072    2557 scope.go:117] "RemoveContainer" containerID="b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92"
Sep 10 21:19:04.277821 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.277802    2557 scope.go:117] "RemoveContainer" containerID="b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92"
Sep 10 21:19:04.278063 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:19:04.278043    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92\": container with ID starting with b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92 not found: ID does not exist" containerID="b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92"
Sep 10 21:19:04.278125 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.278069    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92"} err="failed to get container status \"b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92\": rpc error: code = NotFound desc = could not find container \"b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92\": container with ID starting with b13ba17c1d76ac3dc8d18c8883c20e4875d4ab885905dea995329b0891069a92 not found: ID does not exist"
Sep 10 21:19:04.282049 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.282010    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds" podStartSLOduration=2.281999548 podStartE2EDuration="2.281999548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:04.280146545 +0000 UTC m=+1224.299551863" watchObservedRunningTime="2026-09-10 21:19:04.281999548 +0000 UTC m=+1224.301404866"
Sep 10 21:19:04.291806 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.291784    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90c7c53f-b16e-442e-83c5-1dccb689ec05-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:04.291901 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.291810    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62z2w\" (UniqueName: \"kubernetes.io/projected/90c7c53f-b16e-442e-83c5-1dccb689ec05-kube-api-access-62z2w\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:04.291901 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.291825    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90c7c53f-b16e-442e-83c5-1dccb689ec05-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:19:04.296930 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.296909    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"]
Sep 10 21:19:04.301414 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.301393    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/auth-test-model-755926-kserve-87c44cddc-dbhq4"]
Sep 10 21:19:04.585964 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:04.585931    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90c7c53f-b16e-442e-83c5-1dccb689ec05" path="/var/lib/kubelet/pods/90c7c53f-b16e-442e-83c5-1dccb689ec05/volumes"
Sep 10 21:19:13.059539 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:13.059506    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:13.059949 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:13.059553    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:13.071728 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:13.071703    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:13.321923 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:13.321840    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:19:13.628203 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:13.628124    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:19:32.794013 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.793930    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"]
Sep 10 21:19:32.794882 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.794543    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90c7c53f-b16e-442e-83c5-1dccb689ec05" containerName="main"
Sep 10 21:19:32.794882 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.794563    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="90c7c53f-b16e-442e-83c5-1dccb689ec05" containerName="main"
Sep 10 21:19:32.794882 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.794667    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="90c7c53f-b16e-442e-83c5-1dccb689ec05" containerName="main"
Sep 10 21:19:32.797665 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.797644    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:32.802281 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.802260    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"sub-test-model-755926-kserve-self-signed-certs\""
Sep 10 21:19:32.802988 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.802971    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"openshift-service-ca.crt\""
Sep 10 21:19:32.803302 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.803278    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"kube-root-ca.crt\""
Sep 10 21:19:32.816095 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.816076    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-755926\"/\"default-dockercfg-t6cvs\""
Sep 10 21:19:32.819878 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.819856    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"]
Sep 10 21:19:32.929290 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.929246    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-dshm\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:32.929290 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.929295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-tmp-dir\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:32.929508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.929316    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c667c536-00d2-4716-92fd-a92fa22f536a-tls-certs\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:32.929508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.929370    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-home\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:32.929508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.929431    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2vks\" (UniqueName: \"kubernetes.io/projected/c667c536-00d2-4716-92fd-a92fa22f536a-kube-api-access-n2vks\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:32.929508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:32.929484    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-model-cache\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030228 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030191    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-home\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030258    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2vks\" (UniqueName: \"kubernetes.io/projected/c667c536-00d2-4716-92fd-a92fa22f536a-kube-api-access-n2vks\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030327    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-model-cache\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030394    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-dshm\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030730 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030428    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-tmp-dir\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030730 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030459    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c667c536-00d2-4716-92fd-a92fa22f536a-tls-certs\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030730 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030552    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-home\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030730 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030625    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-model-cache\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.030950 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.030769    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-tmp-dir\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.032608 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.032588    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-dshm\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.032868 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.032850    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c667c536-00d2-4716-92fd-a92fa22f536a-tls-certs\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.039894 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.039875    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2vks\" (UniqueName: \"kubernetes.io/projected/c667c536-00d2-4716-92fd-a92fa22f536a-kube-api-access-n2vks\") pod \"sub-test-model-755926-kserve-5cdc7c78b7-t9492\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") " pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.106954 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.106893    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:33.384422 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.384390    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"]
Sep 10 21:19:33.386227 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:19:33.386199    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc667c536_00d2_4716_92fd_a92fa22f536a.slice/crio-68b5091e788f7c477c119cba18064a717f340fbcb0e00fb8da966ce747ff7407 WatchSource:0}: Error finding container 68b5091e788f7c477c119cba18064a717f340fbcb0e00fb8da966ce747ff7407: Status 404 returned error can't find the container with id 68b5091e788f7c477c119cba18064a717f340fbcb0e00fb8da966ce747ff7407
Sep 10 21:19:33.435137 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:33.435114    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492" event={"ID":"c667c536-00d2-4716-92fd-a92fa22f536a","Type":"ContainerStarted","Data":"68b5091e788f7c477c119cba18064a717f340fbcb0e00fb8da966ce747ff7407"}
Sep 10 21:19:34.442698 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:34.442652    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492" event={"ID":"c667c536-00d2-4716-92fd-a92fa22f536a","Type":"ContainerStarted","Data":"ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb"}
Sep 10 21:19:34.465835 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:34.465790    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492" podStartSLOduration=2.46577519 podStartE2EDuration="2.46577519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:34.464222815 +0000 UTC m=+1254.483628133" watchObservedRunningTime="2026-09-10 21:19:34.46577519 +0000 UTC m=+1254.485180507"
Sep 10 21:19:37.274605 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:37.274563    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:19:43.107940 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:43.107903    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:43.108398 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:43.107952    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:43.120271 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:43.120248    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:43.504364 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:43.504340    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:19:52.126792 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.126757    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"]
Sep 10 21:19:52.131628 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.131611    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.134809 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.134789    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-5a30bd\""
Sep 10 21:19:52.135172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.135158    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-5a30bd-dockercfg-swc79\""
Sep 10 21:19:52.164139 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.164118    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"]
Sep 10 21:19:52.201236 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.201207    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2316bb3-b7e3-4417-9d75-73125d1b8e87-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.201342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.201280    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scqw5\" (UniqueName: \"kubernetes.io/projected/b2316bb3-b7e3-4417-9d75-73125d1b8e87-kube-api-access-scqw5\") pod \"payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.301823 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.301793    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2316bb3-b7e3-4417-9d75-73125d1b8e87-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.301970 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.301933    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scqw5\" (UniqueName: \"kubernetes.io/projected/b2316bb3-b7e3-4417-9d75-73125d1b8e87-kube-api-access-scqw5\") pod \"payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.302602 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.302582    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2316bb3-b7e3-4417-9d75-73125d1b8e87-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.312848 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.312829    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scqw5\" (UniqueName: \"kubernetes.io/projected/b2316bb3-b7e3-4417-9d75-73125d1b8e87-kube-api-access-scqw5\") pod \"payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.442248 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.442184    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:52.729168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:52.729145    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"]
Sep 10 21:19:52.730967 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:19:52.730944    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2316bb3_b7e3_4417_9d75_73125d1b8e87.slice/crio-c0372c4a18dd00416d5ce7c593bee82d03a66525b47a123f13db0d0af613503a WatchSource:0}: Error finding container c0372c4a18dd00416d5ce7c593bee82d03a66525b47a123f13db0d0af613503a: Status 404 returned error can't find the container with id c0372c4a18dd00416d5ce7c593bee82d03a66525b47a123f13db0d0af613503a
Sep 10 21:19:53.554101 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:53.554065    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd" event={"ID":"b2316bb3-b7e3-4417-9d75-73125d1b8e87","Type":"ContainerStarted","Data":"1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028"}
Sep 10 21:19:53.554474 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:53.554151    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd" event={"ID":"b2316bb3-b7e3-4417-9d75-73125d1b8e87","Type":"ContainerStarted","Data":"c0372c4a18dd00416d5ce7c593bee82d03a66525b47a123f13db0d0af613503a"}
Sep 10 21:19:53.554474 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:53.554174    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:19:53.582820 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:19:53.582769    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd" podStartSLOduration=1.582754249 podStartE2EDuration="1.582754249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:53.580428136 +0000 UTC m=+1273.599833480" watchObservedRunningTime="2026-09-10 21:19:53.582754249 +0000 UTC m=+1273.602159563"
Sep 10 21:20:01.139849 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:01.139813    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:20:04.561583 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:04.561547    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:20:21.000583 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.000544    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"]
Sep 10 21:20:21.004008 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.003991    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.011158 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.011139    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a22115-dockercfg-4khdv\""
Sep 10 21:20:21.011562 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.011547    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a22115\""
Sep 10 21:20:21.022273 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.022232    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"]
Sep 10 21:20:21.117037 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.117002    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"]
Sep 10 21:20:21.120461 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.120442    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.139103 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.139081    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"]
Sep 10 21:20:21.158942 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.158906    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxkf5\" (UniqueName: \"kubernetes.io/projected/9ce77a10-a620-4250-9e0f-cf150833425d-kube-api-access-jxkf5\") pod \"payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.159058 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.158993    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce77a10-a620-4250-9e0f-cf150833425d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.259718 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.259615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.259718 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.259646    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5tm7\" (UniqueName: \"kubernetes.io/projected/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-kube-api-access-f5tm7\") pod \"payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.259718 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.259669    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxkf5\" (UniqueName: \"kubernetes.io/projected/9ce77a10-a620-4250-9e0f-cf150833425d-kube-api-access-jxkf5\") pod \"payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.259937 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.259726    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce77a10-a620-4250-9e0f-cf150833425d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.260407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.260388    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce77a10-a620-4250-9e0f-cf150833425d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.270475 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.270447    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxkf5\" (UniqueName: \"kubernetes.io/projected/9ce77a10-a620-4250-9e0f-cf150833425d-kube-api-access-jxkf5\") pod \"payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.314257 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.314240    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.361306 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.361273    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.361443 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.361320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5tm7\" (UniqueName: \"kubernetes.io/projected/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-kube-api-access-f5tm7\") pod \"payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.362216 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.362193    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.374524 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.374469    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5tm7\" (UniqueName: \"kubernetes.io/projected/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-kube-api-access-f5tm7\") pod \"payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") " pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.430813 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.430789    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:21.598750 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.598717    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"]
Sep 10 21:20:21.599244 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:20:21.599222    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce77a10_a620_4250_9e0f_cf150833425d.slice/crio-a19876b719401ab32796b0c9273c0c833be79888628c40d6006a6986276fe36d WatchSource:0}: Error finding container a19876b719401ab32796b0c9273c0c833be79888628c40d6006a6986276fe36d: Status 404 returned error can't find the container with id a19876b719401ab32796b0c9273c0c833be79888628c40d6006a6986276fe36d
Sep 10 21:20:21.710700 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.710534    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"]
Sep 10 21:20:21.710989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.710765    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs" event={"ID":"9ce77a10-a620-4250-9e0f-cf150833425d","Type":"ContainerStarted","Data":"96a51563ee9d6d72ba79a4e488f234c0c245ba9b8a5be95266a39dc4958541c5"}
Sep 10 21:20:21.710989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.710817    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs" event={"ID":"9ce77a10-a620-4250-9e0f-cf150833425d","Type":"ContainerStarted","Data":"a19876b719401ab32796b0c9273c0c833be79888628c40d6006a6986276fe36d"}
Sep 10 21:20:21.710989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.710878    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:21.717088 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:20:21.717058    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df6d4d2_3dbd_4f7d_85f2_d2582f28ea4c.slice/crio-cac6e41cd861e6894ddc3976aba9db1c4e79a4f8d4cf341fa6f298eca560f113 WatchSource:0}: Error finding container cac6e41cd861e6894ddc3976aba9db1c4e79a4f8d4cf341fa6f298eca560f113: Status 404 returned error can't find the container with id cac6e41cd861e6894ddc3976aba9db1c4e79a4f8d4cf341fa6f298eca560f113
Sep 10 21:20:21.735494 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:21.735453    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs" podStartSLOduration=1.7354391900000001 podStartE2EDuration="1.73543919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:21.73330401 +0000 UTC m=+1301.752709333" watchObservedRunningTime="2026-09-10 21:20:21.73543919 +0000 UTC m=+1301.754844507"
Sep 10 21:20:22.717924 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:22.717882    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck" event={"ID":"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c","Type":"ContainerStarted","Data":"486e0019a917c0e1fa55f2d810b64856f7c9213c5832f0ab86eb697f049c5924"}
Sep 10 21:20:22.718354 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:22.717948    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck" event={"ID":"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c","Type":"ContainerStarted","Data":"cac6e41cd861e6894ddc3976aba9db1c4e79a4f8d4cf341fa6f298eca560f113"}
Sep 10 21:20:22.718354 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:22.718162    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:22.806182 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:22.806130    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck" podStartSLOduration=1.806117237 podStartE2EDuration="1.806117237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:22.802143849 +0000 UTC m=+1302.821549166" watchObservedRunningTime="2026-09-10 21:20:22.806117237 +0000 UTC m=+1302.825522555"
Sep 10 21:20:29.050219 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:29.050183    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:20:32.720472 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.720440    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:20:32.842697 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.842652    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7635-d18e11"]
Sep 10 21:20:32.847234 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.847217    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:32.850728 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.850703    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-krpxc\""
Sep 10 21:20:32.856537 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.856514    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7635-d18e11"]
Sep 10 21:20:32.866779 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.866750    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvxk\" (UniqueName: \"kubernetes.io/projected/e5da9348-cb90-4a2f-8a22-90f0dec7b72c-kube-api-access-7lvxk\") pod \"test-curl-7635-d18e11\" (UID: \"e5da9348-cb90-4a2f-8a22-90f0dec7b72c\") " pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:32.968066 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.968023    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvxk\" (UniqueName: \"kubernetes.io/projected/e5da9348-cb90-4a2f-8a22-90f0dec7b72c-kube-api-access-7lvxk\") pod \"test-curl-7635-d18e11\" (UID: \"e5da9348-cb90-4a2f-8a22-90f0dec7b72c\") " pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:32.984864 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:32.984792    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvxk\" (UniqueName: \"kubernetes.io/projected/e5da9348-cb90-4a2f-8a22-90f0dec7b72c-kube-api-access-7lvxk\") pod \"test-curl-7635-d18e11\" (UID: \"e5da9348-cb90-4a2f-8a22-90f0dec7b72c\") " pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:33.157771 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:33.157740    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:33.448157 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:33.442062    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7635-d18e11"]
Sep 10 21:20:33.450734 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:20:33.450677    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5da9348_cb90_4a2f_8a22_90f0dec7b72c.slice/crio-7ab0fe4318cac538ae199163b71dfbab2a3bf551c1241e97405441852987d329 WatchSource:0}: Error finding container 7ab0fe4318cac538ae199163b71dfbab2a3bf551c1241e97405441852987d329: Status 404 returned error can't find the container with id 7ab0fe4318cac538ae199163b71dfbab2a3bf551c1241e97405441852987d329
Sep 10 21:20:33.726293 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:33.726263    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:20:33.781053 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:33.781011    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7635-d18e11" event={"ID":"e5da9348-cb90-4a2f-8a22-90f0dec7b72c","Type":"ContainerStarted","Data":"4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c"}
Sep 10 21:20:33.781234 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:33.781065    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7635-d18e11" event={"ID":"e5da9348-cb90-4a2f-8a22-90f0dec7b72c","Type":"ContainerStarted","Data":"7ab0fe4318cac538ae199163b71dfbab2a3bf551c1241e97405441852987d329"}
Sep 10 21:20:33.810859 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:33.810805    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7635-d18e11" podStartSLOduration=1.810784821 podStartE2EDuration="1.810784821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:33.808790633 +0000 UTC m=+1313.828195952" watchObservedRunningTime="2026-09-10 21:20:33.810784821 +0000 UTC m=+1313.830190141"
Sep 10 21:20:34.669731 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.669618    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7635-d18e11"]
Sep 10 21:20:34.675421 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.675394    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7635-d18e11"]
Sep 10 21:20:34.786653 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:34.786614    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7635-d18e11" container="test-curl-7635-d18e11"
Sep 10 21:20:34.818979 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.818944    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7635-e87a94"]
Sep 10 21:20:34.819400 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.819386    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" containerName="test-curl-7635-d18e11"
Sep 10 21:20:34.819447 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.819401    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" containerName="test-curl-7635-d18e11"
Sep 10 21:20:34.819491 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.819486    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" containerName="test-curl-7635-d18e11"
Sep 10 21:20:34.822914 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.822896    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:34.847556 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.847522    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7635-e87a94"]
Sep 10 21:20:34.884056 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.884027    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr4z4\" (UniqueName: \"kubernetes.io/projected/abfbb803-8cee-4ef2-a55b-a65e3bf5fb35-kube-api-access-qr4z4\") pod \"test-curl-7635-e87a94\" (UID: \"abfbb803-8cee-4ef2-a55b-a65e3bf5fb35\") " pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:34.984463 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:34.984433    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr4z4\" (UniqueName: \"kubernetes.io/projected/abfbb803-8cee-4ef2-a55b-a65e3bf5fb35-kube-api-access-qr4z4\") pod \"test-curl-7635-e87a94\" (UID: \"abfbb803-8cee-4ef2-a55b-a65e3bf5fb35\") " pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:35.001784 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:35.001754    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr4z4\" (UniqueName: \"kubernetes.io/projected/abfbb803-8cee-4ef2-a55b-a65e3bf5fb35-kube-api-access-qr4z4\") pod \"test-curl-7635-e87a94\" (UID: \"abfbb803-8cee-4ef2-a55b-a65e3bf5fb35\") " pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:35.132464 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:35.132436    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:35.415358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:35.414968    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7635-e87a94"]
Sep 10 21:20:35.417867 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:20:35.417840    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabfbb803_8cee_4ef2_a55b_a65e3bf5fb35.slice/crio-45e65f4959ed83dac7ef1a8f1c8943df0b4df6880da6ae62fd71f9eaca352ea2 WatchSource:0}: Error finding container 45e65f4959ed83dac7ef1a8f1c8943df0b4df6880da6ae62fd71f9eaca352ea2: Status 404 returned error can't find the container with id 45e65f4959ed83dac7ef1a8f1c8943df0b4df6880da6ae62fd71f9eaca352ea2
Sep 10 21:20:35.793878 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:35.793839    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7635-e87a94" event={"ID":"abfbb803-8cee-4ef2-a55b-a65e3bf5fb35","Type":"ContainerStarted","Data":"937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f"}
Sep 10 21:20:35.793878 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:35.793891    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7635-e87a94" event={"ID":"abfbb803-8cee-4ef2-a55b-a65e3bf5fb35","Type":"ContainerStarted","Data":"45e65f4959ed83dac7ef1a8f1c8943df0b4df6880da6ae62fd71f9eaca352ea2"}
Sep 10 21:20:35.794329 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:35.793947    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7635-d18e11" podUID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" containerName="test-curl-7635-d18e11" containerID="cri-o://4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c" gracePeriod=2
Sep 10 21:20:35.797737 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:35.797707    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7635-d18e11\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:35.836607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:35.836559    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7635-e87a94" podStartSLOduration=1.8365438950000001 podStartE2EDuration="1.836543895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:35.834922824 +0000 UTC m=+1315.854328143" watchObservedRunningTime="2026-09-10 21:20:35.836543895 +0000 UTC m=+1315.855949250"
Sep 10 21:20:35.841798 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:35.841772    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7635-d18e11\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:36.760471 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:36.760440    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7635-e87a94"]
Sep 10 21:20:36.763991 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:36.763966    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7635-e87a94"]
Sep 10 21:20:36.799592 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:36.799566    2557 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7635-e87a94" container="test-curl-7635-e87a94"
Sep 10 21:20:37.806334 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:37.806301    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7635-e87a94" podUID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" containerName="test-curl-7635-e87a94" containerID="cri-o://937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f" gracePeriod=2
Sep 10 21:20:37.810179 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:37.810146    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7635-e87a94\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:37.994596 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:37.994565    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:38.002906 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:38.002870    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7635-e87a94\" is forbidden: User \"system:node:ip-10-0-140-211.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-211.ec2.internal' and this object" podUID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:38.009212 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.009194    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lvxk\" (UniqueName: \"kubernetes.io/projected/e5da9348-cb90-4a2f-8a22-90f0dec7b72c-kube-api-access-7lvxk\") pod \"e5da9348-cb90-4a2f-8a22-90f0dec7b72c\" (UID: \"e5da9348-cb90-4a2f-8a22-90f0dec7b72c\") "
Sep 10 21:20:38.011339 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.011308    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5da9348-cb90-4a2f-8a22-90f0dec7b72c-kube-api-access-7lvxk" (OuterVolumeSpecName: "kube-api-access-7lvxk") pod "e5da9348-cb90-4a2f-8a22-90f0dec7b72c" (UID: "e5da9348-cb90-4a2f-8a22-90f0dec7b72c"). InnerVolumeSpecName "kube-api-access-7lvxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:38.110451 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.110363    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lvxk\" (UniqueName: \"kubernetes.io/projected/e5da9348-cb90-4a2f-8a22-90f0dec7b72c-kube-api-access-7lvxk\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:38.586786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.586756    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" path="/var/lib/kubelet/pods/e5da9348-cb90-4a2f-8a22-90f0dec7b72c/volumes"
Sep 10 21:20:38.813433 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.813401    2557 generic.go:356] "Generic (PLEG): container finished" podID="e5da9348-cb90-4a2f-8a22-90f0dec7b72c" containerID="4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c" exitCode=137
Sep 10 21:20:38.813881 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.813438    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-d18e11"
Sep 10 21:20:38.813881 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.813504    2557 scope.go:117] "RemoveContainer" containerID="4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c"
Sep 10 21:20:38.822712 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.822693    2557 scope.go:117] "RemoveContainer" containerID="4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c"
Sep 10 21:20:38.822984 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:38.822969    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c\": container with ID starting with 4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c not found: ID does not exist" containerID="4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c"
Sep 10 21:20:38.823037 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.822993    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c"} err="failed to get container status \"4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c\": rpc error: code = NotFound desc = could not find container \"4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c\": container with ID starting with 4d7001ac06f68bffcf8146b4ece1f862cd1305822dbc600c44228b3377d4827c not found: ID does not exist"
Sep 10 21:20:38.912548 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:38.912466    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"]
Sep 10 21:20:39.940223 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:39.940181    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"]
Sep 10 21:20:39.990703 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:39.990656    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:40.025882 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.025854    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr4z4\" (UniqueName: \"kubernetes.io/projected/abfbb803-8cee-4ef2-a55b-a65e3bf5fb35-kube-api-access-qr4z4\") pod \"abfbb803-8cee-4ef2-a55b-a65e3bf5fb35\" (UID: \"abfbb803-8cee-4ef2-a55b-a65e3bf5fb35\") "
Sep 10 21:20:40.027837 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.027811    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfbb803-8cee-4ef2-a55b-a65e3bf5fb35-kube-api-access-qr4z4" (OuterVolumeSpecName: "kube-api-access-qr4z4") pod "abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" (UID: "abfbb803-8cee-4ef2-a55b-a65e3bf5fb35"). InnerVolumeSpecName "kube-api-access-qr4z4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:40.127082 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.127006    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr4z4\" (UniqueName: \"kubernetes.io/projected/abfbb803-8cee-4ef2-a55b-a65e3bf5fb35-kube-api-access-qr4z4\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:40.586795 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.586764    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" path="/var/lib/kubelet/pods/abfbb803-8cee-4ef2-a55b-a65e3bf5fb35/volumes"
Sep 10 21:20:40.627239 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.627214    2557 scope.go:117] "RemoveContainer" containerID="b3d019b9aa9bf4826e3bd14194e306c7b985ee77d4501d67e4403028754e9836"
Sep 10 21:20:40.826657 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.826616    2557 generic.go:356] "Generic (PLEG): container finished" podID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" containerID="937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f" exitCode=137
Sep 10 21:20:40.826840 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.826670    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7635-e87a94"
Sep 10 21:20:40.826840 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.826727    2557 scope.go:117] "RemoveContainer" containerID="937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f"
Sep 10 21:20:40.835363 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.835348    2557 scope.go:117] "RemoveContainer" containerID="937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f"
Sep 10 21:20:40.835600 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:40.835582    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f\": container with ID starting with 937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f not found: ID does not exist" containerID="937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f"
Sep 10 21:20:40.835675 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:40.835608    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f"} err="failed to get container status \"937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f\": rpc error: code = NotFound desc = could not find container \"937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f\": container with ID starting with 937c26306c78234ec65ca660249be4ccd68a24a4bd9bddafde8279bdeaf81c4f not found: ID does not exist"
Sep 10 21:20:43.654916 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:43.654878    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:20:51.138914 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.138884    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"]
Sep 10 21:20:51.139361 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.139312    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" containerName="test-curl-7635-e87a94"
Sep 10 21:20:51.139361 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.139326    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" containerName="test-curl-7635-e87a94"
Sep 10 21:20:51.139465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.139405    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="abfbb803-8cee-4ef2-a55b-a65e3bf5fb35" containerName="test-curl-7635-e87a94"
Sep 10 21:20:51.144187 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.144162    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.146655 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.146630    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-14acb0\"/\"rate-test-model-14acb0-kserve-self-signed-certs\""
Sep 10 21:20:51.153969 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.153946    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"]
Sep 10 21:20:51.226113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.226075    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-home\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.226113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.226118    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tls-certs\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.226358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.226143    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-model-cache\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.226358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.226207    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tmp-dir\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.226358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.226291    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w96m4\" (UniqueName: \"kubernetes.io/projected/f9cf64b3-7ade-4be4-b80b-396cd782ea39-kube-api-access-w96m4\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.226358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.226326    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-dshm\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327258    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w96m4\" (UniqueName: \"kubernetes.io/projected/f9cf64b3-7ade-4be4-b80b-396cd782ea39-kube-api-access-w96m4\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327469 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327313    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-dshm\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327469 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327405    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-home\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327469 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327433    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tls-certs\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327469 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327464    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-model-cache\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327737 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327590    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tmp-dir\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327881 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327862    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-model-cache\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.327934 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327918    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-home\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.328034 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.327997    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tmp-dir\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.329614 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.329592    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-dshm\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.329990 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.329971    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tls-certs\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.337326 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.337302    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w96m4\" (UniqueName: \"kubernetes.io/projected/f9cf64b3-7ade-4be4-b80b-396cd782ea39-kube-api-access-w96m4\") pod \"rate-test-model-14acb0-kserve-58794cd955-rkg6l\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") " pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.453918 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.453823    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:20:51.731127 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.731094    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"]
Sep 10 21:20:51.732325 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:20:51.732299    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9cf64b3_7ade_4be4_b80b_396cd782ea39.slice/crio-4adce7e05708b230ed51bf5faf66140a060d427177a8008bbd797e30d840e564 WatchSource:0}: Error finding container 4adce7e05708b230ed51bf5faf66140a060d427177a8008bbd797e30d840e564: Status 404 returned error can't find the container with id 4adce7e05708b230ed51bf5faf66140a060d427177a8008bbd797e30d840e564
Sep 10 21:20:51.892637 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.892592    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l" event={"ID":"f9cf64b3-7ade-4be4-b80b-396cd782ea39","Type":"ContainerStarted","Data":"af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab"}
Sep 10 21:20:51.892637 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.892641    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l" event={"ID":"f9cf64b3-7ade-4be4-b80b-396cd782ea39","Type":"ContainerStarted","Data":"4adce7e05708b230ed51bf5faf66140a060d427177a8008bbd797e30d840e564"}
Sep 10 21:20:51.914327 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:51.914274    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l" podStartSLOduration=0.914257773 podStartE2EDuration="914.257773ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:51.913803634 +0000 UTC m=+1331.933208956" watchObservedRunningTime="2026-09-10 21:20:51.914257773 +0000 UTC m=+1331.933663138"
Sep 10 21:20:53.931433 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:53.931386    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds" podUID="bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" containerName="main" containerID="cri-o://8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0" gracePeriod=45
Sep 10 21:20:54.193261 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.193204    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:20:54.253978 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.253945    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tmp-dir\") pod \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") "
Sep 10 21:20:54.254147 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.253988    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-model-cache\") pod \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") "
Sep 10 21:20:54.254147 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254027    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-dshm\") pod \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") "
Sep 10 21:20:54.254147 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254113    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-home\") pod \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") "
Sep 10 21:20:54.254147 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254137    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5zrj\" (UniqueName: \"kubernetes.io/projected/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-kube-api-access-f5zrj\") pod \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") "
Sep 10 21:20:54.254335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254162    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tls-certs\") pod \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\" (UID: \"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef\") "
Sep 10 21:20:54.254335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254200    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" (UID: "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:54.254335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254234    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" (UID: "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:54.254454 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254407    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-home" (OuterVolumeSpecName: "home") pod "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" (UID: "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:54.254454 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254447    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:54.254537 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.254464    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:54.256309 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.256278    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-dshm" (OuterVolumeSpecName: "dshm") pod "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" (UID: "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:54.256426 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.256337    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" (UID: "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:20:54.256426 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.256336    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-kube-api-access-f5zrj" (OuterVolumeSpecName: "kube-api-access-f5zrj") pod "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" (UID: "bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef"). InnerVolumeSpecName "kube-api-access-f5zrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:54.355507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.355465    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:54.355507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.355500    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:54.355507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.355511    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5zrj\" (UniqueName: \"kubernetes.io/projected/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-kube-api-access-f5zrj\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:54.355793 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.355521    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:54.910485 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.910395    2557 generic.go:356] "Generic (PLEG): container finished" podID="bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" containerID="8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0" exitCode=0
Sep 10 21:20:54.910485 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.910456    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"
Sep 10 21:20:54.910757 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.910463    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds" event={"ID":"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef","Type":"ContainerDied","Data":"8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0"}
Sep 10 21:20:54.910757 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.910568    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds" event={"ID":"bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef","Type":"ContainerDied","Data":"ae23c05df656c0afc9e3ba68b4846e929b70b4eed80014a341ff988ce71b8dfc"}
Sep 10 21:20:54.910757 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.910590    2557 scope.go:117] "RemoveContainer" containerID="8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0"
Sep 10 21:20:54.919993 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.919973    2557 scope.go:117] "RemoveContainer" containerID="8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0"
Sep 10 21:20:54.920264 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:54.920239    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0\": container with ID starting with 8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0 not found: ID does not exist" containerID="8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0"
Sep 10 21:20:54.920335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.920276    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0"} err="failed to get container status \"8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0\": rpc error: code = NotFound desc = could not find container \"8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0\": container with ID starting with 8c74bee996d9bffcbbb9357301e206c0a746700a7ba72890a99e0f113bed7bd0 not found: ID does not exist"
Sep 10 21:20:54.937676 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.937642    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"]
Sep 10 21:20:54.940048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.940025    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/sub-test-model-14acb0-kserve-7cfcf6cf77-srqds"]
Sep 10 21:20:54.959151 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:54.959119    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492" podUID="c667c536-00d2-4716-92fd-a92fa22f536a" containerName="main" containerID="cri-o://ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb" gracePeriod=45
Sep 10 21:20:55.210280 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.210255    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:20:55.262437 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262408    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-home\") pod \"c667c536-00d2-4716-92fd-a92fa22f536a\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") "
Sep 10 21:20:55.262610 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262445    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-model-cache\") pod \"c667c536-00d2-4716-92fd-a92fa22f536a\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") "
Sep 10 21:20:55.262610 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262515    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-dshm\") pod \"c667c536-00d2-4716-92fd-a92fa22f536a\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") "
Sep 10 21:20:55.262610 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262587    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c667c536-00d2-4716-92fd-a92fa22f536a-tls-certs\") pod \"c667c536-00d2-4716-92fd-a92fa22f536a\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") "
Sep 10 21:20:55.262800 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262654    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2vks\" (UniqueName: \"kubernetes.io/projected/c667c536-00d2-4716-92fd-a92fa22f536a-kube-api-access-n2vks\") pod \"c667c536-00d2-4716-92fd-a92fa22f536a\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") "
Sep 10 21:20:55.262800 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262729    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-model-cache" (OuterVolumeSpecName: "model-cache") pod "c667c536-00d2-4716-92fd-a92fa22f536a" (UID: "c667c536-00d2-4716-92fd-a92fa22f536a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:55.262800 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262745    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-home" (OuterVolumeSpecName: "home") pod "c667c536-00d2-4716-92fd-a92fa22f536a" (UID: "c667c536-00d2-4716-92fd-a92fa22f536a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:55.262800 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262759    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-tmp-dir\") pod \"c667c536-00d2-4716-92fd-a92fa22f536a\" (UID: \"c667c536-00d2-4716-92fd-a92fa22f536a\") "
Sep 10 21:20:55.263051 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.262998    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c667c536-00d2-4716-92fd-a92fa22f536a" (UID: "c667c536-00d2-4716-92fd-a92fa22f536a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:55.263154 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.263139    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:55.263224 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.263156    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:55.263224 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.263165    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:55.264714 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.264669    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-dshm" (OuterVolumeSpecName: "dshm") pod "c667c536-00d2-4716-92fd-a92fa22f536a" (UID: "c667c536-00d2-4716-92fd-a92fa22f536a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:20:55.264883 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.264864    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c667c536-00d2-4716-92fd-a92fa22f536a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c667c536-00d2-4716-92fd-a92fa22f536a" (UID: "c667c536-00d2-4716-92fd-a92fa22f536a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:20:55.265004 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.264985    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c667c536-00d2-4716-92fd-a92fa22f536a-kube-api-access-n2vks" (OuterVolumeSpecName: "kube-api-access-n2vks") pod "c667c536-00d2-4716-92fd-a92fa22f536a" (UID: "c667c536-00d2-4716-92fd-a92fa22f536a"). InnerVolumeSpecName "kube-api-access-n2vks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:20:55.364222 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.364195    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2vks\" (UniqueName: \"kubernetes.io/projected/c667c536-00d2-4716-92fd-a92fa22f536a-kube-api-access-n2vks\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:55.364222 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.364219    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c667c536-00d2-4716-92fd-a92fa22f536a-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:55.364402 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.364229    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c667c536-00d2-4716-92fd-a92fa22f536a-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:20:55.919404 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.919366    2557 generic.go:356] "Generic (PLEG): container finished" podID="c667c536-00d2-4716-92fd-a92fa22f536a" containerID="ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb" exitCode=0
Sep 10 21:20:55.919404 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.919409    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"
Sep 10 21:20:55.919607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.919446    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492" event={"ID":"c667c536-00d2-4716-92fd-a92fa22f536a","Type":"ContainerDied","Data":"ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb"}
Sep 10 21:20:55.919607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.919489    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492" event={"ID":"c667c536-00d2-4716-92fd-a92fa22f536a","Type":"ContainerDied","Data":"68b5091e788f7c477c119cba18064a717f340fbcb0e00fb8da966ce747ff7407"}
Sep 10 21:20:55.919607 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.919505    2557 scope.go:117] "RemoveContainer" containerID="ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb"
Sep 10 21:20:55.928916 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.928899    2557 scope.go:117] "RemoveContainer" containerID="ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb"
Sep 10 21:20:55.929213 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:20:55.929195    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb\": container with ID starting with ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb not found: ID does not exist" containerID="ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb"
Sep 10 21:20:55.929263 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.929221    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb"} err="failed to get container status \"ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb\": rpc error: code = NotFound desc = could not find container \"ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb\": container with ID starting with ae4a443dcaa2d6466e29ad6fdb14ecbe27c2299643c4f8675c022cf4fc640dcb not found: ID does not exist"
Sep 10 21:20:55.946626 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.946604    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"]
Sep 10 21:20:55.952441 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:55.952416    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-755926/sub-test-model-755926-kserve-5cdc7c78b7-t9492"]
Sep 10 21:20:56.586745 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:56.586712    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" path="/var/lib/kubelet/pods/bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef/volumes"
Sep 10 21:20:56.587168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:56.587147    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c667c536-00d2-4716-92fd-a92fa22f536a" path="/var/lib/kubelet/pods/c667c536-00d2-4716-92fd-a92fa22f536a/volumes"
Sep 10 21:20:58.618837 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:20:58.618797    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:21:01.454593 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:01.454556    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:21:01.454593 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:01.454603    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:21:01.466983 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:01.466961    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:21:01.966546 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:01.966518    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:21:03.984440 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:03.984360    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"]
Sep 10 21:21:03.984910 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:03.984559    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd" podUID="b2316bb3-b7e3-4417-9d75-73125d1b8e87" containerName="payload-pre-processing" containerID="cri-o://1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028" gracePeriod=30
Sep 10 21:21:04.232970 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.232941    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:21:04.339586 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.339505    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scqw5\" (UniqueName: \"kubernetes.io/projected/b2316bb3-b7e3-4417-9d75-73125d1b8e87-kube-api-access-scqw5\") pod \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") "
Sep 10 21:21:04.339586 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.339549    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2316bb3-b7e3-4417-9d75-73125d1b8e87-plugins-config-volume\") pod \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\" (UID: \"b2316bb3-b7e3-4417-9d75-73125d1b8e87\") "
Sep 10 21:21:04.340004 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.339979    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2316bb3-b7e3-4417-9d75-73125d1b8e87-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2316bb3-b7e3-4417-9d75-73125d1b8e87" (UID: "b2316bb3-b7e3-4417-9d75-73125d1b8e87"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:04.341570 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.341538    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2316bb3-b7e3-4417-9d75-73125d1b8e87-kube-api-access-scqw5" (OuterVolumeSpecName: "kube-api-access-scqw5") pod "b2316bb3-b7e3-4417-9d75-73125d1b8e87" (UID: "b2316bb3-b7e3-4417-9d75-73125d1b8e87"). InnerVolumeSpecName "kube-api-access-scqw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:04.440885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.440857    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scqw5\" (UniqueName: \"kubernetes.io/projected/b2316bb3-b7e3-4417-9d75-73125d1b8e87-kube-api-access-scqw5\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.440885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.440880    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2316bb3-b7e3-4417-9d75-73125d1b8e87-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:04.975307 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.975271    2557 generic.go:356] "Generic (PLEG): container finished" podID="b2316bb3-b7e3-4417-9d75-73125d1b8e87" containerID="1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028" exitCode=0
Sep 10 21:21:04.975465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.975326    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"
Sep 10 21:21:04.975465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.975351    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd" event={"ID":"b2316bb3-b7e3-4417-9d75-73125d1b8e87","Type":"ContainerDied","Data":"1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028"}
Sep 10 21:21:04.975465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.975424    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd" event={"ID":"b2316bb3-b7e3-4417-9d75-73125d1b8e87","Type":"ContainerDied","Data":"c0372c4a18dd00416d5ce7c593bee82d03a66525b47a123f13db0d0af613503a"}
Sep 10 21:21:04.975465 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.975445    2557 scope.go:117] "RemoveContainer" containerID="1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028"
Sep 10 21:21:04.984573 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.984551    2557 scope.go:117] "RemoveContainer" containerID="1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028"
Sep 10 21:21:04.984876 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:21:04.984828    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028\": container with ID starting with 1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028 not found: ID does not exist" containerID="1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028"
Sep 10 21:21:04.984876 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:04.984852    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028"} err="failed to get container status \"1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028\": rpc error: code = NotFound desc = could not find container \"1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028\": container with ID starting with 1262cb114ef8e3fdf12f178a4937409870c01dc430f6a6561a3c814e4e29c028 not found: ID does not exist"
Sep 10 21:21:05.014135 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:05.014109    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"]
Sep 10 21:21:05.017071 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:05.017049    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5a30bd-6854bd749d-ds8fd"]
Sep 10 21:21:06.586047 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:06.586014    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2316bb3-b7e3-4417-9d75-73125d1b8e87" path="/var/lib/kubelet/pods/b2316bb3-b7e3-4417-9d75-73125d1b8e87/volumes"
Sep 10 21:21:12.267953 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:12.267916    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:21:24.456999 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:24.456965    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:21:27.953932 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:27.953894    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"]
Sep 10 21:21:27.954367 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:27.954084    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck" podUID="1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" containerName="payload-processing" containerID="cri-o://486e0019a917c0e1fa55f2d810b64856f7c9213c5832f0ab86eb697f049c5924" gracePeriod=30
Sep 10 21:21:27.973476 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:27.973447    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"]
Sep 10 21:21:27.973741 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:27.973714    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs" podUID="9ce77a10-a620-4250-9e0f-cf150833425d" containerName="payload-pre-processing" containerID="cri-o://96a51563ee9d6d72ba79a4e488f234c0c245ba9b8a5be95266a39dc4958541c5" gracePeriod=30
Sep 10 21:21:28.115013 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.114977    2557 generic.go:356] "Generic (PLEG): container finished" podID="9ce77a10-a620-4250-9e0f-cf150833425d" containerID="96a51563ee9d6d72ba79a4e488f234c0c245ba9b8a5be95266a39dc4958541c5" exitCode=0
Sep 10 21:21:28.115234 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.115125    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs" event={"ID":"9ce77a10-a620-4250-9e0f-cf150833425d","Type":"ContainerDied","Data":"96a51563ee9d6d72ba79a4e488f234c0c245ba9b8a5be95266a39dc4958541c5"}
Sep 10 21:21:28.117461 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.117432    2557 generic.go:356] "Generic (PLEG): container finished" podID="1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" containerID="486e0019a917c0e1fa55f2d810b64856f7c9213c5832f0ab86eb697f049c5924" exitCode=0
Sep 10 21:21:28.117588 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.117470    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck" event={"ID":"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c","Type":"ContainerDied","Data":"486e0019a917c0e1fa55f2d810b64856f7c9213c5832f0ab86eb697f049c5924"}
Sep 10 21:21:28.248490 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.248469    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:21:28.251464 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.251445    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:21:28.252883 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.252867    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce77a10-a620-4250-9e0f-cf150833425d-plugins-config-volume\") pod \"9ce77a10-a620-4250-9e0f-cf150833425d\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") "
Sep 10 21:21:28.252967 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.252916    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-plugins-config-volume\") pod \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") "
Sep 10 21:21:28.252967 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.252950    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5tm7\" (UniqueName: \"kubernetes.io/projected/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-kube-api-access-f5tm7\") pod \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\" (UID: \"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c\") "
Sep 10 21:21:28.253045 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.252971    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxkf5\" (UniqueName: \"kubernetes.io/projected/9ce77a10-a620-4250-9e0f-cf150833425d-kube-api-access-jxkf5\") pod \"9ce77a10-a620-4250-9e0f-cf150833425d\" (UID: \"9ce77a10-a620-4250-9e0f-cf150833425d\") "
Sep 10 21:21:28.253243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.253221    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce77a10-a620-4250-9e0f-cf150833425d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ce77a10-a620-4250-9e0f-cf150833425d" (UID: "9ce77a10-a620-4250-9e0f-cf150833425d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:28.253331 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.253316    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" (UID: "1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:28.255114 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.255088    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-kube-api-access-f5tm7" (OuterVolumeSpecName: "kube-api-access-f5tm7") pod "1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" (UID: "1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c"). InnerVolumeSpecName "kube-api-access-f5tm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:28.255225 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.255116    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce77a10-a620-4250-9e0f-cf150833425d-kube-api-access-jxkf5" (OuterVolumeSpecName: "kube-api-access-jxkf5") pod "9ce77a10-a620-4250-9e0f-cf150833425d" (UID: "9ce77a10-a620-4250-9e0f-cf150833425d"). InnerVolumeSpecName "kube-api-access-jxkf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:28.353669 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.353632    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5tm7\" (UniqueName: \"kubernetes.io/projected/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-kube-api-access-f5tm7\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:28.353669 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.353661    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxkf5\" (UniqueName: \"kubernetes.io/projected/9ce77a10-a620-4250-9e0f-cf150833425d-kube-api-access-jxkf5\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:28.353669 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.353670    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce77a10-a620-4250-9e0f-cf150833425d-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:28.353936 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:28.353680    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:29.124296 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.124263    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"
Sep 10 21:21:29.124786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.124292    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs" event={"ID":"9ce77a10-a620-4250-9e0f-cf150833425d","Type":"ContainerDied","Data":"a19876b719401ab32796b0c9273c0c833be79888628c40d6006a6986276fe36d"}
Sep 10 21:21:29.124786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.124363    2557 scope.go:117] "RemoveContainer" containerID="96a51563ee9d6d72ba79a4e488f234c0c245ba9b8a5be95266a39dc4958541c5"
Sep 10 21:21:29.125946 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.125818    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck" event={"ID":"1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c","Type":"ContainerDied","Data":"cac6e41cd861e6894ddc3976aba9db1c4e79a4f8d4cf341fa6f298eca560f113"}
Sep 10 21:21:29.125946 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.125852    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"
Sep 10 21:21:29.134702 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.134670    2557 scope.go:117] "RemoveContainer" containerID="486e0019a917c0e1fa55f2d810b64856f7c9213c5832f0ab86eb697f049c5924"
Sep 10 21:21:29.157437 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.157415    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"]
Sep 10 21:21:29.163118 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.163098    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a22115-7d46f6c685-5j7ck"]
Sep 10 21:21:29.187840 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.187820    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"]
Sep 10 21:21:29.197844 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:29.197826    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-a22115-754b7f6d46-58pgs"]
Sep 10 21:21:30.592337 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:30.592305    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" path="/var/lib/kubelet/pods/1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c/volumes"
Sep 10 21:21:30.592723 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:30.592653    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce77a10-a620-4250-9e0f-cf150833425d" path="/var/lib/kubelet/pods/9ce77a10-a620-4250-9e0f-cf150833425d/volumes"
Sep 10 21:21:41.637500 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:41.637465    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:21:47.178190 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.178152    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"]
Sep 10 21:21:47.178779 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.178398    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p" podUID="d90316f7-9818-4ca4-bbf2-dc244ef132f5" containerName="payload-processing" containerID="cri-o://49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248" gracePeriod=30
Sep 10 21:21:47.185827 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.185799    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"]
Sep 10 21:21:47.186024 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.185985    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz" podUID="a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" containerName="payload-pre-processing" containerID="cri-o://57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5" gracePeriod=30
Sep 10 21:21:47.451894 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.451867    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:21:47.454768 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.454749    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:21:47.531029 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.531000    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8hbm\" (UniqueName: \"kubernetes.io/projected/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-kube-api-access-k8hbm\") pod \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") "
Sep 10 21:21:47.531172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.531065    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-plugins-config-volume\") pod \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\" (UID: \"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4\") "
Sep 10 21:21:47.531172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.531106    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf2gz\" (UniqueName: \"kubernetes.io/projected/d90316f7-9818-4ca4-bbf2-dc244ef132f5-kube-api-access-xf2gz\") pod \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") "
Sep 10 21:21:47.531172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.531141    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d90316f7-9818-4ca4-bbf2-dc244ef132f5-plugins-config-volume\") pod \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\" (UID: \"d90316f7-9818-4ca4-bbf2-dc244ef132f5\") "
Sep 10 21:21:47.531454 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.531426    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" (UID: "a1b0da8c-b897-4555-8d8c-8dfd46abf3a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:47.531641 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.531611    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d90316f7-9818-4ca4-bbf2-dc244ef132f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d90316f7-9818-4ca4-bbf2-dc244ef132f5" (UID: "d90316f7-9818-4ca4-bbf2-dc244ef132f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:21:47.533153 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.533134    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d90316f7-9818-4ca4-bbf2-dc244ef132f5-kube-api-access-xf2gz" (OuterVolumeSpecName: "kube-api-access-xf2gz") pod "d90316f7-9818-4ca4-bbf2-dc244ef132f5" (UID: "d90316f7-9818-4ca4-bbf2-dc244ef132f5"). InnerVolumeSpecName "kube-api-access-xf2gz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:47.533220 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.533165    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-kube-api-access-k8hbm" (OuterVolumeSpecName: "kube-api-access-k8hbm") pod "a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" (UID: "a1b0da8c-b897-4555-8d8c-8dfd46abf3a4"). InnerVolumeSpecName "kube-api-access-k8hbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:21:47.632204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.632167    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:47.632204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.632197    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf2gz\" (UniqueName: \"kubernetes.io/projected/d90316f7-9818-4ca4-bbf2-dc244ef132f5-kube-api-access-xf2gz\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:47.632204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.632207    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d90316f7-9818-4ca4-bbf2-dc244ef132f5-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:47.632204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:47.632216    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8hbm\" (UniqueName: \"kubernetes.io/projected/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4-kube-api-access-k8hbm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:21:48.237866 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.237829    2557 generic.go:356] "Generic (PLEG): container finished" podID="a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" containerID="57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5" exitCode=0
Sep 10 21:21:48.238299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.237879    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"
Sep 10 21:21:48.238299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.237911    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz" event={"ID":"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4","Type":"ContainerDied","Data":"57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5"}
Sep 10 21:21:48.238299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.237960    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz" event={"ID":"a1b0da8c-b897-4555-8d8c-8dfd46abf3a4","Type":"ContainerDied","Data":"338c3fc2ad504725a8ebeadd1b4bc2da1c6df80bc01492da250ecf79684259dc"}
Sep 10 21:21:48.238299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.237989    2557 scope.go:117] "RemoveContainer" containerID="57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5"
Sep 10 21:21:48.239351 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.239294    2557 generic.go:356] "Generic (PLEG): container finished" podID="d90316f7-9818-4ca4-bbf2-dc244ef132f5" containerID="49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248" exitCode=0
Sep 10 21:21:48.239351 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.239332    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p" event={"ID":"d90316f7-9818-4ca4-bbf2-dc244ef132f5","Type":"ContainerDied","Data":"49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248"}
Sep 10 21:21:48.239351 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.239329    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"
Sep 10 21:21:48.239538 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.239371    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p" event={"ID":"d90316f7-9818-4ca4-bbf2-dc244ef132f5","Type":"ContainerDied","Data":"140f93caa66130ce97a69762b724e7821e5f83b5dbf1df248cc218c4d34091e9"}
Sep 10 21:21:48.248675 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.248657    2557 scope.go:117] "RemoveContainer" containerID="57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5"
Sep 10 21:21:48.248998 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:21:48.248975    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5\": container with ID starting with 57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5 not found: ID does not exist" containerID="57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5"
Sep 10 21:21:48.249081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.249004    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5"} err="failed to get container status \"57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5\": rpc error: code = NotFound desc = could not find container \"57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5\": container with ID starting with 57314373ddfe11f402413f2a04393f2f578e993855bfb55b91e7281b92bbcef5 not found: ID does not exist"
Sep 10 21:21:48.249081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.249021    2557 scope.go:117] "RemoveContainer" containerID="49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248"
Sep 10 21:21:48.257821 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.257804    2557 scope.go:117] "RemoveContainer" containerID="49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248"
Sep 10 21:21:48.258071 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:21:48.258052    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248\": container with ID starting with 49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248 not found: ID does not exist" containerID="49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248"
Sep 10 21:21:48.258134 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.258083    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248"} err="failed to get container status \"49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248\": rpc error: code = NotFound desc = could not find container \"49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248\": container with ID starting with 49f68396fff18cda2a17bf191c3da877d032d54008df9f27d370fd3c9372a248 not found: ID does not exist"
Sep 10 21:21:48.267146 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.267123    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"]
Sep 10 21:21:48.273485 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.273458    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-290e03-5f5fff78b-wk2vz"]
Sep 10 21:21:48.286018 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.285995    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"]
Sep 10 21:21:48.290292 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.290269    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-290e03-b596dcb55-zbr8p"]
Sep 10 21:21:48.585269 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.585233    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" path="/var/lib/kubelet/pods/a1b0da8c-b897-4555-8d8c-8dfd46abf3a4/volumes"
Sep 10 21:21:48.585662 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:48.585642    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d90316f7-9818-4ca4-bbf2-dc244ef132f5" path="/var/lib/kubelet/pods/d90316f7-9818-4ca4-bbf2-dc244ef132f5/volumes"
Sep 10 21:21:57.974417 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:21:57.974381    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:22:03.490575 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:03.490539    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"]
Sep 10 21:22:05.606937 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:05.606897    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"]
Sep 10 21:22:05.607409 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:05.607146    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2" podUID="530cc2d3-e831-4306-8c34-012b62324050" containerName="maas-api" containerID="cri-o://63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d" gracePeriod=30
Sep 10 21:22:05.850360 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:05.850336    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:22:06.000709 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.000656    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-metrics-certs\") pod \"530cc2d3-e831-4306-8c34-012b62324050\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") "
Sep 10 21:22:06.000866 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.000759    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/530cc2d3-e831-4306-8c34-012b62324050-openshift-service-ca\") pod \"530cc2d3-e831-4306-8c34-012b62324050\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") "
Sep 10 21:22:06.000866 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.000826    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql9rm\" (UniqueName: \"kubernetes.io/projected/530cc2d3-e831-4306-8c34-012b62324050-kube-api-access-ql9rm\") pod \"530cc2d3-e831-4306-8c34-012b62324050\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") "
Sep 10 21:22:06.000957 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.000883    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls\") pod \"530cc2d3-e831-4306-8c34-012b62324050\" (UID: \"530cc2d3-e831-4306-8c34-012b62324050\") "
Sep 10 21:22:06.001186 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.001146    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/530cc2d3-e831-4306-8c34-012b62324050-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "530cc2d3-e831-4306-8c34-012b62324050" (UID: "530cc2d3-e831-4306-8c34-012b62324050"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:22:06.002901 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.002875    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/530cc2d3-e831-4306-8c34-012b62324050-kube-api-access-ql9rm" (OuterVolumeSpecName: "kube-api-access-ql9rm") pod "530cc2d3-e831-4306-8c34-012b62324050" (UID: "530cc2d3-e831-4306-8c34-012b62324050"). InnerVolumeSpecName "kube-api-access-ql9rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:22:06.002993 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.002904    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "530cc2d3-e831-4306-8c34-012b62324050" (UID: "530cc2d3-e831-4306-8c34-012b62324050"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:22:06.002993 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.002930    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "530cc2d3-e831-4306-8c34-012b62324050" (UID: "530cc2d3-e831-4306-8c34-012b62324050"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:22:06.102420 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.102391    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.102420 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.102416    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/530cc2d3-e831-4306-8c34-012b62324050-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.102420 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.102427    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql9rm\" (UniqueName: \"kubernetes.io/projected/530cc2d3-e831-4306-8c34-012b62324050-kube-api-access-ql9rm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.102639 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.102435    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/530cc2d3-e831-4306-8c34-012b62324050-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:06.343779 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.343677    2557 generic.go:356] "Generic (PLEG): container finished" podID="530cc2d3-e831-4306-8c34-012b62324050" containerID="63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d" exitCode=0
Sep 10 21:22:06.343779 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.343746    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"
Sep 10 21:22:06.343958 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.343748    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2" event={"ID":"530cc2d3-e831-4306-8c34-012b62324050","Type":"ContainerDied","Data":"63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d"}
Sep 10 21:22:06.343958 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.343903    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2" event={"ID":"530cc2d3-e831-4306-8c34-012b62324050","Type":"ContainerDied","Data":"b46310c653335196bfe448dd58f2dc169f484c048de30acf6c195de727f6b18a"}
Sep 10 21:22:06.343958 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.343934    2557 scope.go:117] "RemoveContainer" containerID="63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d"
Sep 10 21:22:06.354327 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.354310    2557 scope.go:117] "RemoveContainer" containerID="63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d"
Sep 10 21:22:06.354576 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:22:06.354560    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d\": container with ID starting with 63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d not found: ID does not exist" containerID="63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d"
Sep 10 21:22:06.354618 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.354584    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d"} err="failed to get container status \"63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d\": rpc error: code = NotFound desc = could not find container \"63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d\": container with ID starting with 63f7e33755aebadfa21f3cdd8ff699cc7f2901d79e579b5dc131c791f431990d not found: ID does not exist"
Sep 10 21:22:06.372278 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.372248    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"]
Sep 10 21:22:06.374390 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.374371    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-9f71ac-77b77d96c7-t2vl2"]
Sep 10 21:22:06.586182 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:06.586150    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="530cc2d3-e831-4306-8c34-012b62324050" path="/var/lib/kubelet/pods/530cc2d3-e831-4306-8c34-012b62324050/volumes"
Sep 10 21:22:10.324619 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.324583    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"]
Sep 10 21:22:10.325020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.324963    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" containerName="payload-processing"
Sep 10 21:22:10.325020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.324976    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" containerName="payload-processing"
Sep 10 21:22:10.325020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.324988    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2316bb3-b7e3-4417-9d75-73125d1b8e87" containerName="payload-pre-processing"
Sep 10 21:22:10.325020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.324998    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2316bb3-b7e3-4417-9d75-73125d1b8e87" containerName="payload-pre-processing"
Sep 10 21:22:10.325020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325011    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" containerName="payload-pre-processing"
Sep 10 21:22:10.325020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325018    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" containerName="payload-pre-processing"
Sep 10 21:22:10.325020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325024    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d90316f7-9818-4ca4-bbf2-dc244ef132f5" containerName="payload-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325030    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d90316f7-9818-4ca4-bbf2-dc244ef132f5" containerName="payload-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325041    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c667c536-00d2-4716-92fd-a92fa22f536a" containerName="main"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325046    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c667c536-00d2-4716-92fd-a92fa22f536a" containerName="main"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325053    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce77a10-a620-4250-9e0f-cf150833425d" containerName="payload-pre-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325059    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce77a10-a620-4250-9e0f-cf150833425d" containerName="payload-pre-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325067    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" containerName="main"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325071    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" containerName="main"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325078    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="530cc2d3-e831-4306-8c34-012b62324050" containerName="maas-api"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325083    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="530cc2d3-e831-4306-8c34-012b62324050" containerName="maas-api"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325156    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1df6d4d2-3dbd-4f7d-85f2-d2582f28ea4c" containerName="payload-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325164    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b0da8c-b897-4555-8d8c-8dfd46abf3a4" containerName="payload-pre-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325169    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce77a10-a620-4250-9e0f-cf150833425d" containerName="payload-pre-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325174    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd5bd384-4cd2-4a9d-b9e6-3f6295a8e3ef" containerName="main"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325181    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c667c536-00d2-4716-92fd-a92fa22f536a" containerName="main"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325187    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2316bb3-b7e3-4417-9d75-73125d1b8e87" containerName="payload-pre-processing"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325193    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="530cc2d3-e831-4306-8c34-012b62324050" containerName="maas-api"
Sep 10 21:22:10.325275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.325199    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d90316f7-9818-4ca4-bbf2-dc244ef132f5" containerName="payload-processing"
Sep 10 21:22:10.328302 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.328286    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.331048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.331024    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-84b69f\""
Sep 10 21:22:10.331168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.331129    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-84b69f-dockercfg-r29br\""
Sep 10 21:22:10.336910 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.336883    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6933757a-477d-4746-8a38-a17dd7e30f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.337044 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.336999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz2pm\" (UniqueName: \"kubernetes.io/projected/6933757a-477d-4746-8a38-a17dd7e30f28-kube-api-access-sz2pm\") pod \"payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.340777 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.340751    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"]
Sep 10 21:22:10.394553 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.394524    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"]
Sep 10 21:22:10.398180 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.398163    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.407360 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.407338    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"]
Sep 10 21:22:10.437677 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.437652    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz2pm\" (UniqueName: \"kubernetes.io/projected/6933757a-477d-4746-8a38-a17dd7e30f28-kube-api-access-sz2pm\") pod \"payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.437794 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.437709    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxkqm\" (UniqueName: \"kubernetes.io/projected/e7aca6a2-afc5-4b81-a84c-2639de68d011-kube-api-access-mxkqm\") pod \"payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.437794 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.437745    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7aca6a2-afc5-4b81-a84c-2639de68d011-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.437889 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.437836    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6933757a-477d-4746-8a38-a17dd7e30f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.438479 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.438461    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6933757a-477d-4746-8a38-a17dd7e30f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.447737 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.447715    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz2pm\" (UniqueName: \"kubernetes.io/projected/6933757a-477d-4746-8a38-a17dd7e30f28-kube-api-access-sz2pm\") pod \"payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.538287 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.538257    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxkqm\" (UniqueName: \"kubernetes.io/projected/e7aca6a2-afc5-4b81-a84c-2639de68d011-kube-api-access-mxkqm\") pod \"payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.538423 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.538299    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7aca6a2-afc5-4b81-a84c-2639de68d011-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.538948 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.538930    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7aca6a2-afc5-4b81-a84c-2639de68d011-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.550071 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.550047    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxkqm\" (UniqueName: \"kubernetes.io/projected/e7aca6a2-afc5-4b81-a84c-2639de68d011-kube-api-access-mxkqm\") pod \"payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.638764 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.638675    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:10.709310 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.709275    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:10.932944 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:10.932916    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"]
Sep 10 21:22:10.934111 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:22:10.934090    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6933757a_477d_4746_8a38_a17dd7e30f28.slice/crio-1bf269dcda3f81e957c320cc4b2c71540bf9f43ab3c48c5061a729ea6ebc25bd WatchSource:0}: Error finding container 1bf269dcda3f81e957c320cc4b2c71540bf9f43ab3c48c5061a729ea6ebc25bd: Status 404 returned error can't find the container with id 1bf269dcda3f81e957c320cc4b2c71540bf9f43ab3c48c5061a729ea6ebc25bd
Sep 10 21:22:11.023936 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.023910    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"]
Sep 10 21:22:11.024465 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:22:11.024440    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7aca6a2_afc5_4b81_a84c_2639de68d011.slice/crio-a939706854ab5cd99044baf96e229084998a933498f8baf5b05e52cf9d525632 WatchSource:0}: Error finding container a939706854ab5cd99044baf96e229084998a933498f8baf5b05e52cf9d525632: Status 404 returned error can't find the container with id a939706854ab5cd99044baf96e229084998a933498f8baf5b05e52cf9d525632
Sep 10 21:22:11.375629 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.375589    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82" event={"ID":"e7aca6a2-afc5-4b81-a84c-2639de68d011","Type":"ContainerStarted","Data":"ecbcb5d1230cf10ea443553c328d3f2d7565892d94e4872bedca938bc5c9bc60"}
Sep 10 21:22:11.376096 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.375655    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82" event={"ID":"e7aca6a2-afc5-4b81-a84c-2639de68d011","Type":"ContainerStarted","Data":"a939706854ab5cd99044baf96e229084998a933498f8baf5b05e52cf9d525632"}
Sep 10 21:22:11.376096 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.375709    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:11.378279 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.378252    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4" event={"ID":"6933757a-477d-4746-8a38-a17dd7e30f28","Type":"ContainerStarted","Data":"bced5d9e4b1597a99edd22a883bac8560abe9fd1c5679bed1e1875843e1048b0"}
Sep 10 21:22:11.378410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.378304    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4" event={"ID":"6933757a-477d-4746-8a38-a17dd7e30f28","Type":"ContainerStarted","Data":"1bf269dcda3f81e957c320cc4b2c71540bf9f43ab3c48c5061a729ea6ebc25bd"}
Sep 10 21:22:11.378410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.378347    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:11.399301 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.399256    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82" podStartSLOduration=1.399243045 podStartE2EDuration="1.399243045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:11.397030463 +0000 UTC m=+1411.416435782" watchObservedRunningTime="2026-09-10 21:22:11.399243045 +0000 UTC m=+1411.418648362"
Sep 10 21:22:11.417108 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:11.417061    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4" podStartSLOduration=1.417044045 podStartE2EDuration="1.417044045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:11.415129414 +0000 UTC m=+1411.434534762" watchObservedRunningTime="2026-09-10 21:22:11.417044045 +0000 UTC m=+1411.436449390"
Sep 10 21:22:15.541747 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:15.541707    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:22:18.510895 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.510833    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l" podUID="f9cf64b3-7ade-4be4-b80b-396cd782ea39" containerName="main" containerID="cri-o://af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab" gracePeriod=45
Sep 10 21:22:18.765985 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.765934    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:22:18.805928 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.805904    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w96m4\" (UniqueName: \"kubernetes.io/projected/f9cf64b3-7ade-4be4-b80b-396cd782ea39-kube-api-access-w96m4\") pod \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") "
Sep 10 21:22:18.806067 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.805954    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-model-cache\") pod \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") "
Sep 10 21:22:18.806067 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.805982    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-dshm\") pod \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") "
Sep 10 21:22:18.806067 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806000    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-home\") pod \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") "
Sep 10 21:22:18.806067 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806035    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tls-certs\") pod \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") "
Sep 10 21:22:18.806067 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806058    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tmp-dir\") pod \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\" (UID: \"f9cf64b3-7ade-4be4-b80b-396cd782ea39\") "
Sep 10 21:22:18.806358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806205    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9cf64b3-7ade-4be4-b80b-396cd782ea39" (UID: "f9cf64b3-7ade-4be4-b80b-396cd782ea39"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:18.806358 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806298    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-home" (OuterVolumeSpecName: "home") pod "f9cf64b3-7ade-4be4-b80b-396cd782ea39" (UID: "f9cf64b3-7ade-4be4-b80b-396cd782ea39"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:18.806466 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806398    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:18.806466 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806431    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:18.806466 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.806402    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9cf64b3-7ade-4be4-b80b-396cd782ea39" (UID: "f9cf64b3-7ade-4be4-b80b-396cd782ea39"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:18.808041 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.808009    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-dshm" (OuterVolumeSpecName: "dshm") pod "f9cf64b3-7ade-4be4-b80b-396cd782ea39" (UID: "f9cf64b3-7ade-4be4-b80b-396cd782ea39"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:22:18.808248 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.808231    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9cf64b3-7ade-4be4-b80b-396cd782ea39" (UID: "f9cf64b3-7ade-4be4-b80b-396cd782ea39"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:22:18.808473 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.808452    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9cf64b3-7ade-4be4-b80b-396cd782ea39-kube-api-access-w96m4" (OuterVolumeSpecName: "kube-api-access-w96m4") pod "f9cf64b3-7ade-4be4-b80b-396cd782ea39" (UID: "f9cf64b3-7ade-4be4-b80b-396cd782ea39"). InnerVolumeSpecName "kube-api-access-w96m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:22:18.907249 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.907221    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w96m4\" (UniqueName: \"kubernetes.io/projected/f9cf64b3-7ade-4be4-b80b-396cd782ea39-kube-api-access-w96m4\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:18.907249 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.907244    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:18.907403 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.907253    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:18.907403 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:18.907262    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9cf64b3-7ade-4be4-b80b-396cd782ea39-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:22:19.431812 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.431770    2557 generic.go:356] "Generic (PLEG): container finished" podID="f9cf64b3-7ade-4be4-b80b-396cd782ea39" containerID="af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab" exitCode=0
Sep 10 21:22:19.431812 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.431813    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"
Sep 10 21:22:19.432043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.431823    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l" event={"ID":"f9cf64b3-7ade-4be4-b80b-396cd782ea39","Type":"ContainerDied","Data":"af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab"}
Sep 10 21:22:19.432043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.431868    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l" event={"ID":"f9cf64b3-7ade-4be4-b80b-396cd782ea39","Type":"ContainerDied","Data":"4adce7e05708b230ed51bf5faf66140a060d427177a8008bbd797e30d840e564"}
Sep 10 21:22:19.432043 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.431891    2557 scope.go:117] "RemoveContainer" containerID="af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab"
Sep 10 21:22:19.441149 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.441133    2557 scope.go:117] "RemoveContainer" containerID="af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab"
Sep 10 21:22:19.441403 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:22:19.441386    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab\": container with ID starting with af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab not found: ID does not exist" containerID="af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab"
Sep 10 21:22:19.441482 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.441411    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab"} err="failed to get container status \"af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab\": rpc error: code = NotFound desc = could not find container \"af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab\": container with ID starting with af1a4a567842cf3b64628cb935397b229e755629d2ca27ed345734edb1dd55ab not found: ID does not exist"
Sep 10 21:22:19.457168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.457143    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"]
Sep 10 21:22:19.461162 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:19.461143    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-14acb0/rate-test-model-14acb0-kserve-58794cd955-rkg6l"]
Sep 10 21:22:20.585352 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:20.585320    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9cf64b3-7ade-4be4-b80b-396cd782ea39" path="/var/lib/kubelet/pods/f9cf64b3-7ade-4be4-b80b-396cd782ea39/volumes"
Sep 10 21:22:22.392895 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:22.392866    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:22:22.393254 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:22.393179    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:22:40.085778 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.085700    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"]
Sep 10 21:22:40.086243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.086076    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9cf64b3-7ade-4be4-b80b-396cd782ea39" containerName="main"
Sep 10 21:22:40.086243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.086087    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9cf64b3-7ade-4be4-b80b-396cd782ea39" containerName="main"
Sep 10 21:22:40.086243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.086158    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9cf64b3-7ade-4be4-b80b-396cd782ea39" containerName="main"
Sep 10 21:22:40.090851 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.090832    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.094510 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.094483    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-868994-dockercfg-gq8zd\""
Sep 10 21:22:40.094633 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.094531    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-868994\""
Sep 10 21:22:40.148517 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.148491    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"]
Sep 10 21:22:40.193498 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.193456    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggfbf\" (UniqueName: \"kubernetes.io/projected/175a2a85-1d48-48b3-8726-55d952f5c2d1-kube-api-access-ggfbf\") pod \"payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.193631 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.193576    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a2a85-1d48-48b3-8726-55d952f5c2d1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.294048 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.294018    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggfbf\" (UniqueName: \"kubernetes.io/projected/175a2a85-1d48-48b3-8726-55d952f5c2d1-kube-api-access-ggfbf\") pod \"payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.294168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.294087    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a2a85-1d48-48b3-8726-55d952f5c2d1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.299882 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.299862    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-868994\""
Sep 10 21:22:40.305530 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.305512    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a2a85-1d48-48b3-8726-55d952f5c2d1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.307308 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.307289    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggfbf\" (UniqueName: \"kubernetes.io/projected/175a2a85-1d48-48b3-8726-55d952f5c2d1-kube-api-access-ggfbf\") pod \"payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.403853 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.403778    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-868994-dockercfg-gq8zd\""
Sep 10 21:22:40.411259 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.411235    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:40.696017 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:40.695986    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"]
Sep 10 21:22:40.697775 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:22:40.697742    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175a2a85_1d48_48b3_8726_55d952f5c2d1.slice/crio-6fe1905e07c4cde63acb218fad37fa2a49c47ca1197495b165f9c1fb75d8d453 WatchSource:0}: Error finding container 6fe1905e07c4cde63acb218fad37fa2a49c47ca1197495b165f9c1fb75d8d453: Status 404 returned error can't find the container with id 6fe1905e07c4cde63acb218fad37fa2a49c47ca1197495b165f9c1fb75d8d453
Sep 10 21:22:41.554001 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:41.553966    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57" event={"ID":"175a2a85-1d48-48b3-8726-55d952f5c2d1","Type":"ContainerStarted","Data":"042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8"}
Sep 10 21:22:41.554390 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:41.554024    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57" event={"ID":"175a2a85-1d48-48b3-8726-55d952f5c2d1","Type":"ContainerStarted","Data":"6fe1905e07c4cde63acb218fad37fa2a49c47ca1197495b165f9c1fb75d8d453"}
Sep 10 21:22:41.554390 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:41.554054    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:22:41.582352 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:41.582302    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57" podStartSLOduration=1.5822876099999998 podStartE2EDuration="1.58228761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:22:41.579585502 +0000 UTC m=+1441.598990819" watchObservedRunningTime="2026-09-10 21:22:41.58228761 +0000 UTC m=+1441.601692927"
Sep 10 21:22:45.734786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:45.734746    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:22:52.561165 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:22:52.561130    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:23:07.457184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:07.457155    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-p7hvp_6d437794-4d5d-49d5-8005-31a16c2ee4c3/payload-processing/0.log"
Sep 10 21:23:07.576430 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:07.576403    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57_175a2a85-1d48-48b3-8726-55d952f5c2d1/payload-processing/0.log"
Sep 10 21:23:15.092326 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.092291    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"]
Sep 10 21:23:15.096528 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.096507    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.107129 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.107113    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-84b69f\"/\"openshift-service-ca.crt\""
Sep 10 21:23:15.107205 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.107142    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-84b69f\"/\"kube-root-ca.crt\""
Sep 10 21:23:15.107263 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.107201    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-84b69f\"/\"default-dockercfg-pznfz\""
Sep 10 21:23:15.107313 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.107265    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-84b69f\"/\"ipp-route-84b69f-kserve-self-signed-certs\""
Sep 10 21:23:15.123054 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.123029    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"]
Sep 10 21:23:15.194038 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.194001    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rls9\" (UniqueName: \"kubernetes.io/projected/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-kube-api-access-6rls9\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.194038 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.194042    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tls-certs\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.194255 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.194065    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-model-cache\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.194255 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.194116    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tmp-dir\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.194255 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.194163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-dshm\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.194255 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.194188    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-home\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.294810 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.294774    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tls-certs\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.294810 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.294813    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-model-cache\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.295062 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.294842    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tmp-dir\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.295062 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.294863    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-dshm\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.295062 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.294887    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-home\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.295062 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.294957    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rls9\" (UniqueName: \"kubernetes.io/projected/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-kube-api-access-6rls9\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.295305 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.295276    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-model-cache\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.295305 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.295292    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-home\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.295389 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.295357    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tmp-dir\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.297027 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.297007    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-dshm\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.297302 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.297284    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tls-certs\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.312284 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.312249    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rls9\" (UniqueName: \"kubernetes.io/projected/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-kube-api-access-6rls9\") pod \"ipp-route-84b69f-kserve-5577f44b59-xkcvb\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") " pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.405167 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.405085    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:15.691530 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.691502    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"]
Sep 10 21:23:15.692849 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:23:15.692823    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5f33d65_fbab_4e8d_a6f3_278fed83f9f1.slice/crio-6a406b149f3ecf6f3f9d4cdf9bec1ce5eeb099142b43bd4313fe00a482118aef WatchSource:0}: Error finding container 6a406b149f3ecf6f3f9d4cdf9bec1ce5eeb099142b43bd4313fe00a482118aef: Status 404 returned error can't find the container with id 6a406b149f3ecf6f3f9d4cdf9bec1ce5eeb099142b43bd4313fe00a482118aef
Sep 10 21:23:15.736956 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:15.736920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb" event={"ID":"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1","Type":"ContainerStarted","Data":"6a406b149f3ecf6f3f9d4cdf9bec1ce5eeb099142b43bd4313fe00a482118aef"}
Sep 10 21:23:16.744121 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:16.744086    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb" event={"ID":"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1","Type":"ContainerStarted","Data":"f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2"}
Sep 10 21:23:16.772936 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:16.772883    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb" podStartSLOduration=1.772867557 podStartE2EDuration="1.772867557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:16.771548078 +0000 UTC m=+1476.790953397" watchObservedRunningTime="2026-09-10 21:23:16.772867557 +0000 UTC m=+1476.792272875"
Sep 10 21:23:19.373903 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:19.373869    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:23:25.405564 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:25.405527    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:25.406037 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:25.405586    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:25.418242 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:25.418220    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:25.804064 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:25.804034    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:23:40.525112 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:40.525080    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:23:40.529809 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:40.529786    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:23:42.283721 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:42.283678    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:23:48.429541 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:48.429512    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82_e7aca6a2-afc5-4b81-a84c-2639de68d011/payload-processing/0.log"
Sep 10 21:23:48.554545 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:48.554502    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-p7hvp_6d437794-4d5d-49d5-8005-31a16c2ee4c3/payload-processing/0.log"
Sep 10 21:23:48.693096 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:48.693026    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57_175a2a85-1d48-48b3-8726-55d952f5c2d1/payload-processing/0.log"
Sep 10 21:23:54.735228 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.735185    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"]
Sep 10 21:23:54.738752 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.738735    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:54.743592 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.743568    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-6e4b80-dockercfg-cp9zk\""
Sep 10 21:23:54.743842 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.743824    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-6e4b80\""
Sep 10 21:23:54.761991 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.761968    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"]
Sep 10 21:23:54.765678 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.765654    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:54.767238 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.767218    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"]
Sep 10 21:23:54.794871 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.794844    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"]
Sep 10 21:23:54.838932 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.838905    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:54.839066 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.838935    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxr4j\" (UniqueName: \"kubernetes.io/projected/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-kube-api-access-hxr4j\") pod \"payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:54.939341 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.939313    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff6d41cf-bae5-414a-8687-1e1299852daa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:54.939506 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.939396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:54.939506 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.939414    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8lkm\" (UniqueName: \"kubernetes.io/projected/ff6d41cf-bae5-414a-8687-1e1299852daa-kube-api-access-l8lkm\") pod \"payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:54.939506 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.939432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxr4j\" (UniqueName: \"kubernetes.io/projected/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-kube-api-access-hxr4j\") pod \"payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:54.940099 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.940075    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:54.948486 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:54.948451    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxr4j\" (UniqueName: \"kubernetes.io/projected/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-kube-api-access-hxr4j\") pod \"payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:55.040497 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.040432    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8lkm\" (UniqueName: \"kubernetes.io/projected/ff6d41cf-bae5-414a-8687-1e1299852daa-kube-api-access-l8lkm\") pod \"payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:55.040630 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.040514    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff6d41cf-bae5-414a-8687-1e1299852daa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:55.041338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.041316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff6d41cf-bae5-414a-8687-1e1299852daa-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:55.048994 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.048970    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:55.049758 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.049737    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8lkm\" (UniqueName: \"kubernetes.io/projected/ff6d41cf-bae5-414a-8687-1e1299852daa-kube-api-access-l8lkm\") pod \"payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:55.076059 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.076036    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:55.326472 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.326448    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"]
Sep 10 21:23:55.328175 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:23:55.328151    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd3642e_9e82_4a98_ab5f_0a618d4e131c.slice/crio-1f7467510d1a9faeee4e66fe116958a8ea17f8da06835f34a16c565204c3a908 WatchSource:0}: Error finding container 1f7467510d1a9faeee4e66fe116958a8ea17f8da06835f34a16c565204c3a908: Status 404 returned error can't find the container with id 1f7467510d1a9faeee4e66fe116958a8ea17f8da06835f34a16c565204c3a908
Sep 10 21:23:55.361213 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.361187    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"]
Sep 10 21:23:55.361968 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:23:55.361946    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6d41cf_bae5_414a_8687_1e1299852daa.slice/crio-5e7b5112a6590e0a9ce3d356ab0887ce7be6f151fc6c1befbeea8e0918dfddcf WatchSource:0}: Error finding container 5e7b5112a6590e0a9ce3d356ab0887ce7be6f151fc6c1befbeea8e0918dfddcf: Status 404 returned error can't find the container with id 5e7b5112a6590e0a9ce3d356ab0887ce7be6f151fc6c1befbeea8e0918dfddcf
Sep 10 21:23:55.955103 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.955070    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb" event={"ID":"ff6d41cf-bae5-414a-8687-1e1299852daa","Type":"ContainerStarted","Data":"dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6"}
Sep 10 21:23:55.955550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.955121    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb" event={"ID":"ff6d41cf-bae5-414a-8687-1e1299852daa","Type":"ContainerStarted","Data":"5e7b5112a6590e0a9ce3d356ab0887ce7be6f151fc6c1befbeea8e0918dfddcf"}
Sep 10 21:23:55.955550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.955168    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:23:55.957120 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.957093    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp" event={"ID":"9cd3642e-9e82-4a98-ab5f-0a618d4e131c","Type":"ContainerStarted","Data":"75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8"}
Sep 10 21:23:55.957255 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.957172    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp" event={"ID":"9cd3642e-9e82-4a98-ab5f-0a618d4e131c","Type":"ContainerStarted","Data":"1f7467510d1a9faeee4e66fe116958a8ea17f8da06835f34a16c565204c3a908"}
Sep 10 21:23:55.957255 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.957200    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:23:55.980290 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:55.980238    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb" podStartSLOduration=1.9802201419999998 podStartE2EDuration="1.980220142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:55.976423483 +0000 UTC m=+1515.995828801" watchObservedRunningTime="2026-09-10 21:23:55.980220142 +0000 UTC m=+1515.999625462"
Sep 10 21:23:56.004887 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:23:56.004833    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp" podStartSLOduration=2.004814502 podStartE2EDuration="2.004814502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:23:56.001485564 +0000 UTC m=+1516.020890888" watchObservedRunningTime="2026-09-10 21:23:56.004814502 +0000 UTC m=+1516.024219824"
Sep 10 21:24:02.937118 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:02.937023    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:24:06.965776 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:06.965740    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:24:06.966164 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:06.966130    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:24:24.721268 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:24.721228    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"]
Sep 10 21:24:24.721831 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:24.721478    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb" podUID="ff6d41cf-bae5-414a-8687-1e1299852daa" containerName="payload-processing" containerID="cri-o://dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6" gracePeriod=30
Sep 10 21:24:24.743568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:24.743528    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"]
Sep 10 21:24:24.743968 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:24.743839    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp" podUID="9cd3642e-9e82-4a98-ab5f-0a618d4e131c" containerName="payload-pre-processing" containerID="cri-o://75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8" gracePeriod=30
Sep 10 21:24:25.015512 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.015486    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:24:25.018336 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.018320    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:24:25.108834 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.108806    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxr4j\" (UniqueName: \"kubernetes.io/projected/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-kube-api-access-hxr4j\") pod \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") "
Sep 10 21:24:25.108986 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.108845    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff6d41cf-bae5-414a-8687-1e1299852daa-plugins-config-volume\") pod \"ff6d41cf-bae5-414a-8687-1e1299852daa\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") "
Sep 10 21:24:25.108986 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.108864    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8lkm\" (UniqueName: \"kubernetes.io/projected/ff6d41cf-bae5-414a-8687-1e1299852daa-kube-api-access-l8lkm\") pod \"ff6d41cf-bae5-414a-8687-1e1299852daa\" (UID: \"ff6d41cf-bae5-414a-8687-1e1299852daa\") "
Sep 10 21:24:25.108986 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.108890    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-plugins-config-volume\") pod \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\" (UID: \"9cd3642e-9e82-4a98-ab5f-0a618d4e131c\") "
Sep 10 21:24:25.109291 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.109264    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff6d41cf-bae5-414a-8687-1e1299852daa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff6d41cf-bae5-414a-8687-1e1299852daa" (UID: "ff6d41cf-bae5-414a-8687-1e1299852daa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:25.109382 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.109297    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9cd3642e-9e82-4a98-ab5f-0a618d4e131c" (UID: "9cd3642e-9e82-4a98-ab5f-0a618d4e131c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:25.110853 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.110832    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff6d41cf-bae5-414a-8687-1e1299852daa-kube-api-access-l8lkm" (OuterVolumeSpecName: "kube-api-access-l8lkm") pod "ff6d41cf-bae5-414a-8687-1e1299852daa" (UID: "ff6d41cf-bae5-414a-8687-1e1299852daa"). InnerVolumeSpecName "kube-api-access-l8lkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:25.110944 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.110924    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-kube-api-access-hxr4j" (OuterVolumeSpecName: "kube-api-access-hxr4j") pod "9cd3642e-9e82-4a98-ab5f-0a618d4e131c" (UID: "9cd3642e-9e82-4a98-ab5f-0a618d4e131c"). InnerVolumeSpecName "kube-api-access-hxr4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:25.124274 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.124250    2557 generic.go:356] "Generic (PLEG): container finished" podID="ff6d41cf-bae5-414a-8687-1e1299852daa" containerID="dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6" exitCode=0
Sep 10 21:24:25.124391 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.124297    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"
Sep 10 21:24:25.124391 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.124291    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb" event={"ID":"ff6d41cf-bae5-414a-8687-1e1299852daa","Type":"ContainerDied","Data":"dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6"}
Sep 10 21:24:25.124495 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.124442    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb" event={"ID":"ff6d41cf-bae5-414a-8687-1e1299852daa","Type":"ContainerDied","Data":"5e7b5112a6590e0a9ce3d356ab0887ce7be6f151fc6c1befbeea8e0918dfddcf"}
Sep 10 21:24:25.124495 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.124472    2557 scope.go:117] "RemoveContainer" containerID="dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6"
Sep 10 21:24:25.125801 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.125780    2557 generic.go:356] "Generic (PLEG): container finished" podID="9cd3642e-9e82-4a98-ab5f-0a618d4e131c" containerID="75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8" exitCode=0
Sep 10 21:24:25.125906 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.125833    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"
Sep 10 21:24:25.125906 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.125857    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp" event={"ID":"9cd3642e-9e82-4a98-ab5f-0a618d4e131c","Type":"ContainerDied","Data":"75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8"}
Sep 10 21:24:25.125906 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.125893    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp" event={"ID":"9cd3642e-9e82-4a98-ab5f-0a618d4e131c","Type":"ContainerDied","Data":"1f7467510d1a9faeee4e66fe116958a8ea17f8da06835f34a16c565204c3a908"}
Sep 10 21:24:25.135135 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.135116    2557 scope.go:117] "RemoveContainer" containerID="dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6"
Sep 10 21:24:25.135446 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:24:25.135426    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6\": container with ID starting with dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6 not found: ID does not exist" containerID="dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6"
Sep 10 21:24:25.135496 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.135468    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6"} err="failed to get container status \"dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6\": rpc error: code = NotFound desc = could not find container \"dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6\": container with ID starting with dd18c1f320858b8758a97b67c0f575e278867765ae1277ab7bdb5fb90f1696a6 not found: ID does not exist"
Sep 10 21:24:25.135496 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.135484    2557 scope.go:117] "RemoveContainer" containerID="75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8"
Sep 10 21:24:25.143989 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.143969    2557 scope.go:117] "RemoveContainer" containerID="75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8"
Sep 10 21:24:25.144259 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:24:25.144239    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8\": container with ID starting with 75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8 not found: ID does not exist" containerID="75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8"
Sep 10 21:24:25.144320 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.144270    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8"} err="failed to get container status \"75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8\": rpc error: code = NotFound desc = could not find container \"75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8\": container with ID starting with 75505320207ef6b2f8328846c51ca3e8e84c67b75ce313d6fa3fcde61d1c4ed8 not found: ID does not exist"
Sep 10 21:24:25.151498 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.151472    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"]
Sep 10 21:24:25.153933 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.153910    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6e4b80-688946dcf8-twpmp"]
Sep 10 21:24:25.166124 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.166097    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"]
Sep 10 21:24:25.171903 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.171879    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6e4b80-fdc8859df-p6rcb"]
Sep 10 21:24:25.210279 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.210250    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxr4j\" (UniqueName: \"kubernetes.io/projected/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-kube-api-access-hxr4j\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:25.210366 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.210285    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff6d41cf-bae5-414a-8687-1e1299852daa-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:25.210366 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.210302    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8lkm\" (UniqueName: \"kubernetes.io/projected/ff6d41cf-bae5-414a-8687-1e1299852daa-kube-api-access-l8lkm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:25.210366 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:25.210316    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9cd3642e-9e82-4a98-ab5f-0a618d4e131c-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:26.585319 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:26.585286    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd3642e-9e82-4a98-ab5f-0a618d4e131c" path="/var/lib/kubelet/pods/9cd3642e-9e82-4a98-ab5f-0a618d4e131c/volumes"
Sep 10 21:24:26.585663 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:26.585630    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff6d41cf-bae5-414a-8687-1e1299852daa" path="/var/lib/kubelet/pods/ff6d41cf-bae5-414a-8687-1e1299852daa/volumes"
Sep 10 21:24:29.345577 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:29.345540    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:24:48.539750 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:48.539712    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:24:51.052210 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.052176    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"]
Sep 10 21:24:51.052660 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.052451    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82" podUID="e7aca6a2-afc5-4b81-a84c-2639de68d011" containerName="payload-processing" containerID="cri-o://ecbcb5d1230cf10ea443553c328d3f2d7565892d94e4872bedca938bc5c9bc60" gracePeriod=30
Sep 10 21:24:51.060612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.060245    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"]
Sep 10 21:24:51.060612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.060498    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4" podUID="6933757a-477d-4746-8a38-a17dd7e30f28" containerName="payload-pre-processing" containerID="cri-o://bced5d9e4b1597a99edd22a883bac8560abe9fd1c5679bed1e1875843e1048b0" gracePeriod=30
Sep 10 21:24:51.275306 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.275279    2557 generic.go:356] "Generic (PLEG): container finished" podID="6933757a-477d-4746-8a38-a17dd7e30f28" containerID="bced5d9e4b1597a99edd22a883bac8560abe9fd1c5679bed1e1875843e1048b0" exitCode=0
Sep 10 21:24:51.275459 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.275355    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4" event={"ID":"6933757a-477d-4746-8a38-a17dd7e30f28","Type":"ContainerDied","Data":"bced5d9e4b1597a99edd22a883bac8560abe9fd1c5679bed1e1875843e1048b0"}
Sep 10 21:24:51.277258 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.277235    2557 generic.go:356] "Generic (PLEG): container finished" podID="e7aca6a2-afc5-4b81-a84c-2639de68d011" containerID="ecbcb5d1230cf10ea443553c328d3f2d7565892d94e4872bedca938bc5c9bc60" exitCode=0
Sep 10 21:24:51.277366 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.277291    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82" event={"ID":"e7aca6a2-afc5-4b81-a84c-2639de68d011","Type":"ContainerDied","Data":"ecbcb5d1230cf10ea443553c328d3f2d7565892d94e4872bedca938bc5c9bc60"}
Sep 10 21:24:51.332728 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.332707    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:24:51.336184 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.336131    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:24:51.433340 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.433314    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxkqm\" (UniqueName: \"kubernetes.io/projected/e7aca6a2-afc5-4b81-a84c-2639de68d011-kube-api-access-mxkqm\") pod \"e7aca6a2-afc5-4b81-a84c-2639de68d011\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") "
Sep 10 21:24:51.433508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.433357    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sz2pm\" (UniqueName: \"kubernetes.io/projected/6933757a-477d-4746-8a38-a17dd7e30f28-kube-api-access-sz2pm\") pod \"6933757a-477d-4746-8a38-a17dd7e30f28\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") "
Sep 10 21:24:51.433508 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.433391    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7aca6a2-afc5-4b81-a84c-2639de68d011-plugins-config-volume\") pod \"e7aca6a2-afc5-4b81-a84c-2639de68d011\" (UID: \"e7aca6a2-afc5-4b81-a84c-2639de68d011\") "
Sep 10 21:24:51.433630 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.433510    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6933757a-477d-4746-8a38-a17dd7e30f28-plugins-config-volume\") pod \"6933757a-477d-4746-8a38-a17dd7e30f28\" (UID: \"6933757a-477d-4746-8a38-a17dd7e30f28\") "
Sep 10 21:24:51.433817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.433794    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7aca6a2-afc5-4b81-a84c-2639de68d011-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e7aca6a2-afc5-4b81-a84c-2639de68d011" (UID: "e7aca6a2-afc5-4b81-a84c-2639de68d011"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:51.433898 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.433883    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7aca6a2-afc5-4b81-a84c-2639de68d011-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:51.433971 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.433906    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6933757a-477d-4746-8a38-a17dd7e30f28-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6933757a-477d-4746-8a38-a17dd7e30f28" (UID: "6933757a-477d-4746-8a38-a17dd7e30f28"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:24:51.435452 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.435416    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7aca6a2-afc5-4b81-a84c-2639de68d011-kube-api-access-mxkqm" (OuterVolumeSpecName: "kube-api-access-mxkqm") pod "e7aca6a2-afc5-4b81-a84c-2639de68d011" (UID: "e7aca6a2-afc5-4b81-a84c-2639de68d011"). InnerVolumeSpecName "kube-api-access-mxkqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:51.435452 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.435430    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6933757a-477d-4746-8a38-a17dd7e30f28-kube-api-access-sz2pm" (OuterVolumeSpecName: "kube-api-access-sz2pm") pod "6933757a-477d-4746-8a38-a17dd7e30f28" (UID: "6933757a-477d-4746-8a38-a17dd7e30f28"). InnerVolumeSpecName "kube-api-access-sz2pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:24:51.534912 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.534866    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxkqm\" (UniqueName: \"kubernetes.io/projected/e7aca6a2-afc5-4b81-a84c-2639de68d011-kube-api-access-mxkqm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:51.534912 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.534905    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz2pm\" (UniqueName: \"kubernetes.io/projected/6933757a-477d-4746-8a38-a17dd7e30f28-kube-api-access-sz2pm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:51.534912 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:51.534920    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6933757a-477d-4746-8a38-a17dd7e30f28-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:24:52.283529 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.283496    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"
Sep 10 21:24:52.283529 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.283522    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4" event={"ID":"6933757a-477d-4746-8a38-a17dd7e30f28","Type":"ContainerDied","Data":"1bf269dcda3f81e957c320cc4b2c71540bf9f43ab3c48c5061a729ea6ebc25bd"}
Sep 10 21:24:52.284031 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.283583    2557 scope.go:117] "RemoveContainer" containerID="bced5d9e4b1597a99edd22a883bac8560abe9fd1c5679bed1e1875843e1048b0"
Sep 10 21:24:52.284927 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.284907    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"
Sep 10 21:24:52.284995 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.284947    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82" event={"ID":"e7aca6a2-afc5-4b81-a84c-2639de68d011","Type":"ContainerDied","Data":"a939706854ab5cd99044baf96e229084998a933498f8baf5b05e52cf9d525632"}
Sep 10 21:24:52.292564 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.292546    2557 scope.go:117] "RemoveContainer" containerID="ecbcb5d1230cf10ea443553c328d3f2d7565892d94e4872bedca938bc5c9bc60"
Sep 10 21:24:52.311627 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.311603    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"]
Sep 10 21:24:52.318137 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.318115    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-84b69f-6597c7db58-9rgl4"]
Sep 10 21:24:52.329234 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.329189    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"]
Sep 10 21:24:52.332699 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.332662    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-84b69f-5fbddc774c-x6z82"]
Sep 10 21:24:52.585642 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.585561    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6933757a-477d-4746-8a38-a17dd7e30f28" path="/var/lib/kubelet/pods/6933757a-477d-4746-8a38-a17dd7e30f28/volumes"
Sep 10 21:24:52.586041 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:52.586020    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7aca6a2-afc5-4b81-a84c-2639de68d011" path="/var/lib/kubelet/pods/e7aca6a2-afc5-4b81-a84c-2639de68d011/volumes"
Sep 10 21:24:56.609072 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:24:56.609043    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"]
Sep 10 21:25:01.634745 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:01.634706    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:25:11.628379 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:11.628331    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb" podUID="d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" containerName="main" containerID="cri-o://f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2" gracePeriod=45
Sep 10 21:25:11.882765 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:11.882704    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:25:12.004291 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004250    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tmp-dir\") pod \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") "
Sep 10 21:25:12.004491 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004307    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-dshm\") pod \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") "
Sep 10 21:25:12.004491 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004351    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rls9\" (UniqueName: \"kubernetes.io/projected/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-kube-api-access-6rls9\") pod \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") "
Sep 10 21:25:12.004491 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004396    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tls-certs\") pod \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") "
Sep 10 21:25:12.004491 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004436    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-model-cache\") pod \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") "
Sep 10 21:25:12.004491 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004487    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-home\") pod \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\" (UID: \"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1\") "
Sep 10 21:25:12.004799 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004604    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" (UID: "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:12.004857 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004805    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" (UID: "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:12.004911 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004869    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:12.004954 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.004892    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-home" (OuterVolumeSpecName: "home") pod "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" (UID: "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:12.006580 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.006558    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-dshm" (OuterVolumeSpecName: "dshm") pod "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" (UID: "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:25:12.006675 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.006654    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-kube-api-access-6rls9" (OuterVolumeSpecName: "kube-api-access-6rls9") pod "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" (UID: "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1"). InnerVolumeSpecName "kube-api-access-6rls9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:12.006732 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.006669    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" (UID: "d5f33d65-fbab-4e8d-a6f3-278fed83f9f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:25:12.106170 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.106132    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:12.106170 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.106162    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rls9\" (UniqueName: \"kubernetes.io/projected/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-kube-api-access-6rls9\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:12.106170 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.106177    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:12.106407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.106190    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:12.106407 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.106200    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:12.392957 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.392923    2557 generic.go:356] "Generic (PLEG): container finished" podID="d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" containerID="f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2" exitCode=0
Sep 10 21:25:12.393122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.392965    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"
Sep 10 21:25:12.393122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.393010    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb" event={"ID":"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1","Type":"ContainerDied","Data":"f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2"}
Sep 10 21:25:12.393122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.393056    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb" event={"ID":"d5f33d65-fbab-4e8d-a6f3-278fed83f9f1","Type":"ContainerDied","Data":"6a406b149f3ecf6f3f9d4cdf9bec1ce5eeb099142b43bd4313fe00a482118aef"}
Sep 10 21:25:12.393122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.393077    2557 scope.go:117] "RemoveContainer" containerID="f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2"
Sep 10 21:25:12.402920 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.402898    2557 scope.go:117] "RemoveContainer" containerID="f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2"
Sep 10 21:25:12.403173 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:25:12.403153    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2\": container with ID starting with f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2 not found: ID does not exist" containerID="f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2"
Sep 10 21:25:12.403236 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.403180    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2"} err="failed to get container status \"f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2\": rpc error: code = NotFound desc = could not find container \"f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2\": container with ID starting with f2a4bb62a4f19ec07f643a48714d1fe59ddfad3a04c893f8e905f9812580fcd2 not found: ID does not exist"
Sep 10 21:25:12.422526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.422499    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"]
Sep 10 21:25:12.424669 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.424649    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-84b69f/ipp-route-84b69f-kserve-5577f44b59-xkcvb"]
Sep 10 21:25:12.588325 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:12.588292    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" path="/var/lib/kubelet/pods/d5f33d65-fbab-4e8d-a6f3-278fed83f9f1/volumes"
Sep 10 21:25:52.247328 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.247291    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"]
Sep 10 21:25:52.247859 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.247549    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57" podUID="175a2a85-1d48-48b3-8726-55d952f5c2d1" containerName="payload-processing" containerID="cri-o://042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8" gracePeriod=30
Sep 10 21:25:52.489098 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.489073    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:25:52.569195 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.569115    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a2a85-1d48-48b3-8726-55d952f5c2d1-plugins-config-volume\") pod \"175a2a85-1d48-48b3-8726-55d952f5c2d1\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") "
Sep 10 21:25:52.569361 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.569199    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggfbf\" (UniqueName: \"kubernetes.io/projected/175a2a85-1d48-48b3-8726-55d952f5c2d1-kube-api-access-ggfbf\") pod \"175a2a85-1d48-48b3-8726-55d952f5c2d1\" (UID: \"175a2a85-1d48-48b3-8726-55d952f5c2d1\") "
Sep 10 21:25:52.569543 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.569516    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/175a2a85-1d48-48b3-8726-55d952f5c2d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "175a2a85-1d48-48b3-8726-55d952f5c2d1" (UID: "175a2a85-1d48-48b3-8726-55d952f5c2d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:25:52.571252 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.571219    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/175a2a85-1d48-48b3-8726-55d952f5c2d1-kube-api-access-ggfbf" (OuterVolumeSpecName: "kube-api-access-ggfbf") pod "175a2a85-1d48-48b3-8726-55d952f5c2d1" (UID: "175a2a85-1d48-48b3-8726-55d952f5c2d1"). InnerVolumeSpecName "kube-api-access-ggfbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:25:52.619963 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.619932    2557 generic.go:356] "Generic (PLEG): container finished" podID="175a2a85-1d48-48b3-8726-55d952f5c2d1" containerID="042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8" exitCode=0
Sep 10 21:25:52.620094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.619974    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"
Sep 10 21:25:52.620094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.620022    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57" event={"ID":"175a2a85-1d48-48b3-8726-55d952f5c2d1","Type":"ContainerDied","Data":"042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8"}
Sep 10 21:25:52.620094 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.620070    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57" event={"ID":"175a2a85-1d48-48b3-8726-55d952f5c2d1","Type":"ContainerDied","Data":"6fe1905e07c4cde63acb218fad37fa2a49c47ca1197495b165f9c1fb75d8d453"}
Sep 10 21:25:52.620221 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.620096    2557 scope.go:117] "RemoveContainer" containerID="042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8"
Sep 10 21:25:52.629143 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.629126    2557 scope.go:117] "RemoveContainer" containerID="042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8"
Sep 10 21:25:52.629416 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:25:52.629398    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8\": container with ID starting with 042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8 not found: ID does not exist" containerID="042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8"
Sep 10 21:25:52.629470 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.629425    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8"} err="failed to get container status \"042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8\": rpc error: code = NotFound desc = could not find container \"042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8\": container with ID starting with 042784b2b549dea150b4e7f673911e4be9e919b51947b76a8b3f7fb5b92300f8 not found: ID does not exist"
Sep 10 21:25:52.639390 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.639346    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"]
Sep 10 21:25:52.642572 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.642548    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-868994-57bcb6cb95-x4b57"]
Sep 10 21:25:52.669918 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.669891    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a2a85-1d48-48b3-8726-55d952f5c2d1-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:52.669918 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:52.669915    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggfbf\" (UniqueName: \"kubernetes.io/projected/175a2a85-1d48-48b3-8726-55d952f5c2d1-kube-api-access-ggfbf\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:25:54.585763 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:54.585724    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="175a2a85-1d48-48b3-8726-55d952f5c2d1" path="/var/lib/kubelet/pods/175a2a85-1d48-48b3-8726-55d952f5c2d1/volumes"
Sep 10 21:25:55.546439 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:25:55.546394    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:26:11.063326 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.063293    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"]
Sep 10 21:26:11.063845 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.063548    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q" podUID="c5438a58-1869-4579-8263-cb498e4be0bf" containerName="payload-processing" containerID="cri-o://8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2" gracePeriod=30
Sep 10 21:26:11.327109 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.327040    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:26:11.435637 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.435603    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpnwm\" (UniqueName: \"kubernetes.io/projected/c5438a58-1869-4579-8263-cb498e4be0bf-kube-api-access-vpnwm\") pod \"c5438a58-1869-4579-8263-cb498e4be0bf\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") "
Sep 10 21:26:11.435817 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.435678    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5438a58-1869-4579-8263-cb498e4be0bf-plugins-config-volume\") pod \"c5438a58-1869-4579-8263-cb498e4be0bf\" (UID: \"c5438a58-1869-4579-8263-cb498e4be0bf\") "
Sep 10 21:26:11.436063 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.436041    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5438a58-1869-4579-8263-cb498e4be0bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c5438a58-1869-4579-8263-cb498e4be0bf" (UID: "c5438a58-1869-4579-8263-cb498e4be0bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:11.437696 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.437666    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5438a58-1869-4579-8263-cb498e4be0bf-kube-api-access-vpnwm" (OuterVolumeSpecName: "kube-api-access-vpnwm") pod "c5438a58-1869-4579-8263-cb498e4be0bf" (UID: "c5438a58-1869-4579-8263-cb498e4be0bf"). InnerVolumeSpecName "kube-api-access-vpnwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:11.537336 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.537304    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpnwm\" (UniqueName: \"kubernetes.io/projected/c5438a58-1869-4579-8263-cb498e4be0bf-kube-api-access-vpnwm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.537336 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.537342    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5438a58-1869-4579-8263-cb498e4be0bf-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:26:11.724623 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.724587    2557 generic.go:356] "Generic (PLEG): container finished" podID="c5438a58-1869-4579-8263-cb498e4be0bf" containerID="8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2" exitCode=0
Sep 10 21:26:11.724786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.724632    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"
Sep 10 21:26:11.724786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.724666    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q" event={"ID":"c5438a58-1869-4579-8263-cb498e4be0bf","Type":"ContainerDied","Data":"8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2"}
Sep 10 21:26:11.724786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.724738    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q" event={"ID":"c5438a58-1869-4579-8263-cb498e4be0bf","Type":"ContainerDied","Data":"4bc2004e08277b84e52d3b6f576daf2cc5461ecdc1babb8daf7d7f4c97bf2cf5"}
Sep 10 21:26:11.724786 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.724761    2557 scope.go:117] "RemoveContainer" containerID="8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2"
Sep 10 21:26:11.733961 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.733943    2557 scope.go:117] "RemoveContainer" containerID="8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2"
Sep 10 21:26:11.734219 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:26:11.734202    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2\": container with ID starting with 8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2 not found: ID does not exist" containerID="8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2"
Sep 10 21:26:11.734267 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.734226    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2"} err="failed to get container status \"8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2\": rpc error: code = NotFound desc = could not find container \"8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2\": container with ID starting with 8b85f5529af617693323c4955e6ae200993e6f7878df41c8d4a1b720c4de4bc2 not found: ID does not exist"
Sep 10 21:26:11.747829 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.747808    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"]
Sep 10 21:26:11.752345 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:11.752324    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-14acb0-798dbf95f7-qfs4q"]
Sep 10 21:26:12.594795 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:12.594759    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5438a58-1869-4579-8263-cb498e4be0bf" path="/var/lib/kubelet/pods/c5438a58-1869-4579-8263-cb498e4be0bf/volumes"
Sep 10 21:26:14.231550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:14.231513    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:26:29.109224 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.109189    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"]
Sep 10 21:26:29.109727 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.109406    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt" podUID="affb1540-b933-4758-b413-2435a249bb47" containerName="payload-processing" containerID="cri-o://c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe" gracePeriod=30
Sep 10 21:26:29.354612 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.354584    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:26:29.390119 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.390046    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz92h\" (UniqueName: \"kubernetes.io/projected/affb1540-b933-4758-b413-2435a249bb47-kube-api-access-hz92h\") pod \"affb1540-b933-4758-b413-2435a249bb47\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") "
Sep 10 21:26:29.390119 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.390113    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/affb1540-b933-4758-b413-2435a249bb47-plugins-config-volume\") pod \"affb1540-b933-4758-b413-2435a249bb47\" (UID: \"affb1540-b933-4758-b413-2435a249bb47\") "
Sep 10 21:26:29.390473 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.390448    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/affb1540-b933-4758-b413-2435a249bb47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "affb1540-b933-4758-b413-2435a249bb47" (UID: "affb1540-b933-4758-b413-2435a249bb47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:26:29.392169 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.392133    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/affb1540-b933-4758-b413-2435a249bb47-kube-api-access-hz92h" (OuterVolumeSpecName: "kube-api-access-hz92h") pod "affb1540-b933-4758-b413-2435a249bb47" (UID: "affb1540-b933-4758-b413-2435a249bb47"). InnerVolumeSpecName "kube-api-access-hz92h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:26:29.491742 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.491709    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/affb1540-b933-4758-b413-2435a249bb47-plugins-config-volume\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.491742 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.491739    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz92h\" (UniqueName: \"kubernetes.io/projected/affb1540-b933-4758-b413-2435a249bb47-kube-api-access-hz92h\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:26:29.822430 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.822389    2557 generic.go:356] "Generic (PLEG): container finished" podID="affb1540-b933-4758-b413-2435a249bb47" containerID="c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe" exitCode=0
Sep 10 21:26:29.822430 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.822433    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"
Sep 10 21:26:29.822638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.822472    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt" event={"ID":"affb1540-b933-4758-b413-2435a249bb47","Type":"ContainerDied","Data":"c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe"}
Sep 10 21:26:29.822638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.822529    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt" event={"ID":"affb1540-b933-4758-b413-2435a249bb47","Type":"ContainerDied","Data":"1b6b66ab6862c91e7a23e1086702563e9a8f2d723d38427d2458eaf253a63d2b"}
Sep 10 21:26:29.822638 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.822553    2557 scope.go:117] "RemoveContainer" containerID="c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe"
Sep 10 21:26:29.832153 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.832134    2557 scope.go:117] "RemoveContainer" containerID="c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe"
Sep 10 21:26:29.832429 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:26:29.832412    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe\": container with ID starting with c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe not found: ID does not exist" containerID="c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe"
Sep 10 21:26:29.832493 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.832443    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe"} err="failed to get container status \"c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe\": rpc error: code = NotFound desc = could not find container \"c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe\": container with ID starting with c739ffd81643d14eeeeb9b88d53b437e2a1b45e87724a55c766757f7616a30fe not found: ID does not exist"
Sep 10 21:26:29.845199 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.845172    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"]
Sep 10 21:26:29.849405 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:29.849382    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-755926-5b55c7db8f-257dt"]
Sep 10 21:26:30.585570 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:30.585539    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="affb1540-b933-4758-b413-2435a249bb47" path="/var/lib/kubelet/pods/affb1540-b933-4758-b413-2435a249bb47/volumes"
Sep 10 21:26:31.932810 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:31.932775    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:26:41.246897 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:26:41.246859    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:28:03.141754 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:28:03.141715    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:28:30.139724 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:28:30.139678    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:28:37.943617 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:28:37.943536    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:28:40.561698 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:28:40.561654    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:28:40.568293 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:28:40.568272    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:28:45.040417 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:28:45.040384    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:28:52.441738 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:28:52.441696    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:29:06.429310 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:29:06.429263    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:29:16.431627 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:29:16.431589    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:30:44.737400 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:30:44.737357    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:30:52.928127 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:30:52.928090    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:31:01.539015 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:01.538979    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:31:11.103824 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.103782    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"]
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104140    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6933757a-477d-4746-8a38-a17dd7e30f28" containerName="payload-pre-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104152    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6933757a-477d-4746-8a38-a17dd7e30f28" containerName="payload-pre-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104162    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7aca6a2-afc5-4b81-a84c-2639de68d011" containerName="payload-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104167    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7aca6a2-afc5-4b81-a84c-2639de68d011" containerName="payload-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104177    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="affb1540-b933-4758-b413-2435a249bb47" containerName="payload-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104182    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="affb1540-b933-4758-b413-2435a249bb47" containerName="payload-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104192    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175a2a85-1d48-48b3-8726-55d952f5c2d1" containerName="payload-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104197    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="175a2a85-1d48-48b3-8726-55d952f5c2d1" containerName="payload-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104203    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd3642e-9e82-4a98-ab5f-0a618d4e131c" containerName="payload-pre-processing"
Sep 10 21:31:11.104209 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104209    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd3642e-9e82-4a98-ab5f-0a618d4e131c" containerName="payload-pre-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104217    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff6d41cf-bae5-414a-8687-1e1299852daa" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104222    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff6d41cf-bae5-414a-8687-1e1299852daa" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104235    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" containerName="main"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104241    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" containerName="main"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104247    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5438a58-1869-4579-8263-cb498e4be0bf" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104252    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5438a58-1869-4579-8263-cb498e4be0bf" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104305    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5438a58-1869-4579-8263-cb498e4be0bf" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104312    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff6d41cf-bae5-414a-8687-1e1299852daa" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104317    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="affb1540-b933-4758-b413-2435a249bb47" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104324    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6933757a-477d-4746-8a38-a17dd7e30f28" containerName="payload-pre-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104330    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd3642e-9e82-4a98-ab5f-0a618d4e131c" containerName="payload-pre-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104337    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f33d65-fbab-4e8d-a6f3-278fed83f9f1" containerName="main"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104344    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7aca6a2-afc5-4b81-a84c-2639de68d011" containerName="payload-processing"
Sep 10 21:31:11.104526 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.104350    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="175a2a85-1d48-48b3-8726-55d952f5c2d1" containerName="payload-processing"
Sep 10 21:31:11.107379 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.107360    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.112793 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.112768    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-650a5a7f-kserve-self-signed-certs\""
Sep 10 21:31:11.125923 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.125875    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"]
Sep 10 21:31:11.163899 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.163859    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d46w\" (UniqueName: \"kubernetes.io/projected/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-kube-api-access-5d46w\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.163899 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.163898    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-dshm\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.164133 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.163921    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tmp-dir\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.164133 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.164007    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tls-certs\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.164133 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.164084    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-model-cache\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.164133 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.164104    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-home\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.264784 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.264744    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d46w\" (UniqueName: \"kubernetes.io/projected/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-kube-api-access-5d46w\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.264784 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.264790    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-dshm\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.265011 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.264821    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tmp-dir\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.265011 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.264878    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tls-certs\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.265011 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.264962    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-model-cache\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.265182 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.265104    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-home\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.265268 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.265248    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tmp-dir\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.265374 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.265354    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-model-cache\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.265420 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.265393    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-home\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.267204 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.267177    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-dshm\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.267385 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.267370    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tls-certs\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.285372 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.285340    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d46w\" (UniqueName: \"kubernetes.io/projected/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-kube-api-access-5d46w\") pod \"e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") " pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.416243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.416140    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:11.705518 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:11.705493    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"]
Sep 10 21:31:11.706928 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:31:11.706887    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b6546e5_d4d1_41c9_ad1e_05a025c11bb8.slice/crio-0cb9b2178d7f429981f5f062307907f7f735b06153a760f9455c082eab654d9f WatchSource:0}: Error finding container 0cb9b2178d7f429981f5f062307907f7f735b06153a760f9455c082eab654d9f: Status 404 returned error can't find the container with id 0cb9b2178d7f429981f5f062307907f7f735b06153a760f9455c082eab654d9f
Sep 10 21:31:12.322598 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:12.322558    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j" event={"ID":"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8","Type":"ContainerStarted","Data":"fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190"}
Sep 10 21:31:12.322598 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:12.322604    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j" event={"ID":"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8","Type":"ContainerStarted","Data":"0cb9b2178d7f429981f5f062307907f7f735b06153a760f9455c082eab654d9f"}
Sep 10 21:31:12.343170 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:12.343114    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j" podStartSLOduration=1.343097973 podStartE2EDuration="1.343097973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:31:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:31:12.340482772 +0000 UTC m=+1952.359888202" watchObservedRunningTime="2026-09-10 21:31:12.343097973 +0000 UTC m=+1952.362503282"
Sep 10 21:31:13.531175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:13.531141    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:31:21.416540 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:21.416483    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:21.417067 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:21.416559    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:21.429410 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:21.429382    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:22.388138 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:22.388108    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:31:26.443201 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:31:26.443153    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:32:08.356391 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:08.356353    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"]
Sep 10 21:32:10.655527 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:10.655491    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:32:19.526117 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:19.526082    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:32:23.375537 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.375487    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j" podUID="6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" containerName="main" containerID="cri-o://fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190" gracePeriod=45
Sep 10 21:32:23.619844 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.619820    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:32:23.698187 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.698116    2557 generic.go:356] "Generic (PLEG): container finished" podID="6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" containerID="fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190" exitCode=0
Sep 10 21:32:23.698187 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.698153    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"
Sep 10 21:32:23.698342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.698201    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j" event={"ID":"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8","Type":"ContainerDied","Data":"fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190"}
Sep 10 21:32:23.698342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.698239    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j" event={"ID":"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8","Type":"ContainerDied","Data":"0cb9b2178d7f429981f5f062307907f7f735b06153a760f9455c082eab654d9f"}
Sep 10 21:32:23.698342 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.698257    2557 scope.go:117] "RemoveContainer" containerID="fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190"
Sep 10 21:32:23.706966 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.706949    2557 scope.go:117] "RemoveContainer" containerID="fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190"
Sep 10 21:32:23.707230 ip-10-0-140-211 kubenswrapper[2557]: E0910 21:32:23.707201    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190\": container with ID starting with fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190 not found: ID does not exist" containerID="fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190"
Sep 10 21:32:23.707299 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.707223    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190"} err="failed to get container status \"fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190\": rpc error: code = NotFound desc = could not find container \"fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190\": container with ID starting with fdcabf6f315e6884280f9c112718a111933443564148363e57e484ca8b3bd190 not found: ID does not exist"
Sep 10 21:32:23.708440 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708426    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-home\") pod \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") "
Sep 10 21:32:23.708507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708453    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-dshm\") pod \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") "
Sep 10 21:32:23.708507 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708487    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tls-certs\") pod \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") "
Sep 10 21:32:23.708595 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708544    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-model-cache\") pod \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") "
Sep 10 21:32:23.708595 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708575    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d46w\" (UniqueName: \"kubernetes.io/projected/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-kube-api-access-5d46w\") pod \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") "
Sep 10 21:32:23.708715 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708598    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tmp-dir\") pod \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\" (UID: \"6b6546e5-d4d1-41c9-ad1e-05a025c11bb8\") "
Sep 10 21:32:23.708715 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708677    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-home" (OuterVolumeSpecName: "home") pod "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" (UID: "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:23.708885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708812    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" (UID: "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:23.708885 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708846    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-home\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:23.709010 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.708986    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" (UID: "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:23.710664 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.710644    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-dshm" (OuterVolumeSpecName: "dshm") pod "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" (UID: "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:23.710664 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.710655    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" (UID: "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:23.710832 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.710667    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-kube-api-access-5d46w" (OuterVolumeSpecName: "kube-api-access-5d46w") pod "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" (UID: "6b6546e5-d4d1-41c9-ad1e-05a025c11bb8"). InnerVolumeSpecName "kube-api-access-5d46w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:23.809792 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.809770    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d46w\" (UniqueName: \"kubernetes.io/projected/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-kube-api-access-5d46w\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:23.809792 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.809790    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tmp-dir\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:23.809917 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.809799    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-dshm\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:23.809917 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.809807    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-tls-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:23.809917 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:23.809815    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8-model-cache\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:24.022131 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:24.022103    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"]
Sep 10 21:32:24.027973 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:24.027951    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-650a5a7f-kserve-7b95b55899-mdm4j"]
Sep 10 21:32:24.585195 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:24.585159    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" path="/var/lib/kubelet/pods/6b6546e5-d4d1-41c9-ad1e-05a025c11bb8/volumes"
Sep 10 21:32:25.836769 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:25.836733    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:32:33.032170 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.032137    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:32:33.331333 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.331248    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"]
Sep 10 21:32:33.331671 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.331658    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" containerName="main"
Sep 10 21:32:33.331671 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.331673    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" containerName="main"
Sep 10 21:32:33.331801 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.331769    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b6546e5-d4d1-41c9-ad1e-05a025c11bb8" containerName="main"
Sep 10 21:32:33.347792 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.347747    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"]
Sep 10 21:32:33.348009 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.347866    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.351749 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.351723    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 21:32:33.351749 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.351740    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:32:33.351927 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.351779    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 21:32:33.351927 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.351723    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8cgpb\""
Sep 10 21:32:33.351927 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.351840    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 21:32:33.490081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.490046    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx6cg\" (UniqueName: \"kubernetes.io/projected/b4525225-3e29-4e5f-b64e-d1c65b9d899a-kube-api-access-lx6cg\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.490081 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.490086    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4525225-3e29-4e5f-b64e-d1c65b9d899a-openshift-service-ca\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.490282 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.490111    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-metrics-certs\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.490282 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.490205    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-maas-api-tls\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.591387 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.591299    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4525225-3e29-4e5f-b64e-d1c65b9d899a-openshift-service-ca\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.591387 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.591341    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-metrics-certs\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.591568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.591449    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-maas-api-tls\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.591568 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.591517    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx6cg\" (UniqueName: \"kubernetes.io/projected/b4525225-3e29-4e5f-b64e-d1c65b9d899a-kube-api-access-lx6cg\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.592007 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.591990    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4525225-3e29-4e5f-b64e-d1c65b9d899a-openshift-service-ca\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.593861 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.593839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-metrics-certs\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.593968 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.593878    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-maas-api-tls\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.600275 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.600251    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx6cg\" (UniqueName: \"kubernetes.io/projected/b4525225-3e29-4e5f-b64e-d1c65b9d899a-kube-api-access-lx6cg\") pod \"maas-api-7f7f8945f9-nqk46\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") " pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.659718 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.659668    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:33.931901 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.931828    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"]
Sep 10 21:32:33.935260 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:32:33.935232    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4525225_3e29_4e5f_b64e_d1c65b9d899a.slice/crio-c5fbc6f54c2866b22658c7952103d2d14e68d2a263281bcf0390300468fe27f9 WatchSource:0}: Error finding container c5fbc6f54c2866b22658c7952103d2d14e68d2a263281bcf0390300468fe27f9: Status 404 returned error can't find the container with id c5fbc6f54c2866b22658c7952103d2d14e68d2a263281bcf0390300468fe27f9
Sep 10 21:32:33.936906 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:33.936883    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:32:34.758374 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:34.758340    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46" event={"ID":"b4525225-3e29-4e5f-b64e-d1c65b9d899a","Type":"ContainerStarted","Data":"3e5334c70c53ec32c67e60e8bec324d22181050ef6e4689e01c6a79f202dfaa0"}
Sep 10 21:32:34.758829 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:34.758395    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46" event={"ID":"b4525225-3e29-4e5f-b64e-d1c65b9d899a","Type":"ContainerStarted","Data":"c5fbc6f54c2866b22658c7952103d2d14e68d2a263281bcf0390300468fe27f9"}
Sep 10 21:32:34.758829 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:34.758425    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:34.779712 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:34.779647    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46" podStartSLOduration=1.39272883 podStartE2EDuration="1.779627845s" totalImagesPullingTime="386.899015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:34.777728433 +0000 UTC m=+2034.797133751" watchObservedRunningTime="2026-09-10 21:32:34.779627845 +0000 UTC m=+2034.799033165"
Sep 10 21:32:40.597414 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.597385    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"]
Sep 10 21:32:40.597949 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.597562    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46" podUID="b4525225-3e29-4e5f-b64e-d1c65b9d899a" containerName="maas-api" containerID="cri-o://3e5334c70c53ec32c67e60e8bec324d22181050ef6e4689e01c6a79f202dfaa0" gracePeriod=30
Sep 10 21:32:40.602484 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.602463    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:40.637557 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.637534    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-jcbsb"]
Sep 10 21:32:40.640993 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.640975    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.657350 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.657323    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-jcbsb"]
Sep 10 21:32:40.755140 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.755109    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60781eb6-6b64-454d-8302-04b5ebd213d8-metrics-certs\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.755312 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.755171    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60781eb6-6b64-454d-8302-04b5ebd213d8-openshift-service-ca\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.755312 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.755212    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60781eb6-6b64-454d-8302-04b5ebd213d8-maas-api-tls\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.755312 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.755239    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhfbw\" (UniqueName: \"kubernetes.io/projected/60781eb6-6b64-454d-8302-04b5ebd213d8-kube-api-access-mhfbw\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.793663 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.793629    2557 generic.go:356] "Generic (PLEG): container finished" podID="b4525225-3e29-4e5f-b64e-d1c65b9d899a" containerID="3e5334c70c53ec32c67e60e8bec324d22181050ef6e4689e01c6a79f202dfaa0" exitCode=0
Sep 10 21:32:40.793818 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.793718    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46" event={"ID":"b4525225-3e29-4e5f-b64e-d1c65b9d899a","Type":"ContainerDied","Data":"3e5334c70c53ec32c67e60e8bec324d22181050ef6e4689e01c6a79f202dfaa0"}
Sep 10 21:32:40.830657 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.830629    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:32:40.836898 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.836879    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:40.855941 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.855922    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60781eb6-6b64-454d-8302-04b5ebd213d8-metrics-certs\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.856039 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.855959    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60781eb6-6b64-454d-8302-04b5ebd213d8-openshift-service-ca\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.856039 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.855985    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60781eb6-6b64-454d-8302-04b5ebd213d8-maas-api-tls\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.856039 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.856009    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhfbw\" (UniqueName: \"kubernetes.io/projected/60781eb6-6b64-454d-8302-04b5ebd213d8-kube-api-access-mhfbw\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.856825 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.856800    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60781eb6-6b64-454d-8302-04b5ebd213d8-openshift-service-ca\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.859152 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.859038    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60781eb6-6b64-454d-8302-04b5ebd213d8-metrics-certs\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.859152 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.859109    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60781eb6-6b64-454d-8302-04b5ebd213d8-maas-api-tls\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.865006 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.864957    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhfbw\" (UniqueName: \"kubernetes.io/projected/60781eb6-6b64-454d-8302-04b5ebd213d8-kube-api-access-mhfbw\") pod \"maas-api-559477f47-jcbsb\" (UID: \"60781eb6-6b64-454d-8302-04b5ebd213d8\") " pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:40.959045 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.959016    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-maas-api-tls\") pod \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") "
Sep 10 21:32:40.959185 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.959063    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx6cg\" (UniqueName: \"kubernetes.io/projected/b4525225-3e29-4e5f-b64e-d1c65b9d899a-kube-api-access-lx6cg\") pod \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") "
Sep 10 21:32:40.959185 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.959084    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-metrics-certs\") pod \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") "
Sep 10 21:32:40.959185 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.959118    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4525225-3e29-4e5f-b64e-d1c65b9d899a-openshift-service-ca\") pod \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\" (UID: \"b4525225-3e29-4e5f-b64e-d1c65b9d899a\") "
Sep 10 21:32:40.959934 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.959903    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4525225-3e29-4e5f-b64e-d1c65b9d899a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4525225-3e29-4e5f-b64e-d1c65b9d899a" (UID: "b4525225-3e29-4e5f-b64e-d1c65b9d899a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:32:40.961909 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.961883    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4525225-3e29-4e5f-b64e-d1c65b9d899a-kube-api-access-lx6cg" (OuterVolumeSpecName: "kube-api-access-lx6cg") pod "b4525225-3e29-4e5f-b64e-d1c65b9d899a" (UID: "b4525225-3e29-4e5f-b64e-d1c65b9d899a"). InnerVolumeSpecName "kube-api-access-lx6cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:40.962037 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.962014    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4525225-3e29-4e5f-b64e-d1c65b9d899a" (UID: "b4525225-3e29-4e5f-b64e-d1c65b9d899a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:40.962226 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.962200    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b4525225-3e29-4e5f-b64e-d1c65b9d899a" (UID: "b4525225-3e29-4e5f-b64e-d1c65b9d899a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:32:40.980448 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:40.980425    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:41.060243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.060217    2557 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-metrics-certs\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:41.060243 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.060242    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4525225-3e29-4e5f-b64e-d1c65b9d899a-openshift-service-ca\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:41.060393 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.060253    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4525225-3e29-4e5f-b64e-d1c65b9d899a-maas-api-tls\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:41.060393 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.060263    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx6cg\" (UniqueName: \"kubernetes.io/projected/b4525225-3e29-4e5f-b64e-d1c65b9d899a-kube-api-access-lx6cg\") on node \"ip-10-0-140-211.ec2.internal\" DevicePath \"\""
Sep 10 21:32:41.253353 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.253324    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-559477f47-jcbsb"]
Sep 10 21:32:41.254747 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:32:41.254721    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60781eb6_6b64_454d_8302_04b5ebd213d8.slice/crio-524e7bc64dc2649cd5b34bb02a29e98e6762260b8db0608997f7ff8ad11c06ca WatchSource:0}: Error finding container 524e7bc64dc2649cd5b34bb02a29e98e6762260b8db0608997f7ff8ad11c06ca: Status 404 returned error can't find the container with id 524e7bc64dc2649cd5b34bb02a29e98e6762260b8db0608997f7ff8ad11c06ca
Sep 10 21:32:41.800937 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.800894    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb" event={"ID":"60781eb6-6b64-454d-8302-04b5ebd213d8","Type":"ContainerStarted","Data":"061d28bed9b881bc5f5234adabbe8e287deda7ef310cfaa119e06ce8784f54eb"}
Sep 10 21:32:41.801383 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.800969    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:41.801383 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.800994    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb" event={"ID":"60781eb6-6b64-454d-8302-04b5ebd213d8","Type":"ContainerStarted","Data":"524e7bc64dc2649cd5b34bb02a29e98e6762260b8db0608997f7ff8ad11c06ca"}
Sep 10 21:32:41.802431 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.802402    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46" event={"ID":"b4525225-3e29-4e5f-b64e-d1c65b9d899a","Type":"ContainerDied","Data":"c5fbc6f54c2866b22658c7952103d2d14e68d2a263281bcf0390300468fe27f9"}
Sep 10 21:32:41.802550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.802438    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"
Sep 10 21:32:41.802550 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.802462    2557 scope.go:117] "RemoveContainer" containerID="3e5334c70c53ec32c67e60e8bec324d22181050ef6e4689e01c6a79f202dfaa0"
Sep 10 21:32:41.820372 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.820323    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb" podStartSLOduration=1.48626991 podStartE2EDuration="1.820302711s" totalImagesPullingTime="334.032801ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:41.818296184 +0000 UTC m=+2041.837701525" watchObservedRunningTime="2026-09-10 21:32:41.820302711 +0000 UTC m=+2041.839708030"
Sep 10 21:32:41.834345 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.834314    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"]
Sep 10 21:32:41.838173 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:41.838152    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7f7f8945f9-nqk46"]
Sep 10 21:32:42.586514 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:42.586470    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4525225-3e29-4e5f-b64e-d1c65b9d899a" path="/var/lib/kubelet/pods/b4525225-3e29-4e5f-b64e-d1c65b9d899a/volumes"
Sep 10 21:32:47.530417 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:47.530377    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:32:47.814020 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:47.813946    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-559477f47-jcbsb"
Sep 10 21:32:53.431149 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:32:53.431116    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:33:00.125009 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:00.124972    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-xqb2j"]
Sep 10 21:33:02.935112 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:02.935007    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b76ccd69c-zrxgn_e3f63195-83b8-4fbe-91e6-afbce0e443be/authorino/0.log"
Sep 10 21:33:08.676124 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:08.676095    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-2xnnx_1d8de411-a529-4246-9c4c-309d9bdb4e07/copy-manifests/0.log"
Sep 10 21:33:08.697558 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:08.697532    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-2xnnx_1d8de411-a529-4246-9c4c-309d9bdb4e07/manager/0.log"
Sep 10 21:33:09.373095 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:09.373064    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-559477f47-jcbsb_60781eb6-6b64-454d-8302-04b5ebd213d8/maas-api/0.log"
Sep 10 21:33:10.445105 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:10.445069    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b76ccd69c-zrxgn_e3f63195-83b8-4fbe-91e6-afbce0e443be/authorino/0.log"
Sep 10 21:33:11.035186 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:11.035156    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-xqb2j_4e95b1d0-713a-4951-8fa3-42800a2629f5/limitador/0.log"
Sep 10 21:33:11.834513 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:11.834484    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5tts4_ae9a0726-e680-4910-a364-c5f46d94210b/kube-auth-proxy/0.log"
Sep 10 21:33:12.193984 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:12.193899    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-p7hvp_6d437794-4d5d-49d5-8005-31a16c2ee4c3/payload-processing/0.log"
Sep 10 21:33:12.641041 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:12.641016    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-lfgfq_2f5b07ce-c298-46b4-af80-2a1c8ad29f36/main/0.log"
Sep 10 21:33:13.299118 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:13.299087    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-sb6j5_04738a95-2480-47cd-adfd-ce3e954f9156/main/0.log"
Sep 10 21:33:21.291076 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:21.291042    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6chtx_5d19bacb-b122-4c14-8f38-99989bc66f71/global-pull-secret-syncer/0.log"
Sep 10 21:33:21.533338 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:21.533298    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pjjhk_80156b77-2422-4f5e-93fa-cc6a4878e15b/konnectivity-agent/0.log"
Sep 10 21:33:21.610110 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:21.610019    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-211.ec2.internal_42033061b2efc1a5de4a9db1dfbac058/haproxy/0.log"
Sep 10 21:33:26.516068 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:26.516030    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b76ccd69c-zrxgn_e3f63195-83b8-4fbe-91e6-afbce0e443be/authorino/0.log"
Sep 10 21:33:26.873464 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:26.873373    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-xqb2j_4e95b1d0-713a-4951-8fa3-42800a2629f5/limitador/0.log"
Sep 10 21:33:28.486783 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:28.486753    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-ghgxz_0fdee397-ee70-4c4f-87e3-95289083c592/cluster-monitoring-operator/0.log"
Sep 10 21:33:28.529115 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:28.529084    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-rhkk4_62ba902d-4465-48d9-b9c8-d907423c9ee8/kube-state-metrics/0.log"
Sep 10 21:33:28.548547 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:28.548522    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-rhkk4_62ba902d-4465-48d9-b9c8-d907423c9ee8/kube-rbac-proxy-main/0.log"
Sep 10 21:33:28.588172 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:28.588124    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-rhkk4_62ba902d-4465-48d9-b9c8-d907423c9ee8/kube-rbac-proxy-self/0.log"
Sep 10 21:33:28.721499 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:28.721469    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5c4zz_3f4c0123-ee19-4773-89af-479ff47e1562/node-exporter/0.log"
Sep 10 21:33:28.753689 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:28.753663    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5c4zz_3f4c0123-ee19-4773-89af-479ff47e1562/kube-rbac-proxy/0.log"
Sep 10 21:33:28.792122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:28.792093    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5c4zz_3f4c0123-ee19-4773-89af-479ff47e1562/init-textfile/0.log"
Sep 10 21:33:29.009200 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.009120    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-45spn_e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74/kube-rbac-proxy-main/0.log"
Sep 10 21:33:29.032413 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.032385    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-45spn_e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74/kube-rbac-proxy-self/0.log"
Sep 10 21:33:29.055351 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.055328    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-45spn_e998b1ee-76b5-4fe5-a2ea-d14fd0c43b74/openshift-state-metrics/0.log"
Sep 10 21:33:29.316604 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.316519    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-l2qn9_3648aac7-e065-4b16-b52b-66fe5641da9a/prometheus-operator/0.log"
Sep 10 21:33:29.336291 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.336261    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-l2qn9_3648aac7-e065-4b16-b52b-66fe5641da9a/kube-rbac-proxy/0.log"
Sep 10 21:33:29.362923 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.362903    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-zr49p_6d069a24-c52a-4e12-8bcd-edc94386a876/prometheus-operator-admission-webhook/0.log"
Sep 10 21:33:29.399601 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.399578    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78f977ff8f-krggd_d65230cd-1a5d-44e2-8d16-216d529dc190/telemeter-client/0.log"
Sep 10 21:33:29.422198 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.422177    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78f977ff8f-krggd_d65230cd-1a5d-44e2-8d16-216d529dc190/reload/0.log"
Sep 10 21:33:29.444674 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:29.444648    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78f977ff8f-krggd_d65230cd-1a5d-44e2-8d16-216d529dc190/kube-rbac-proxy/0.log"
Sep 10 21:33:31.369723 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.369675    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"]
Sep 10 21:33:31.370122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.370050    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4525225-3e29-4e5f-b64e-d1c65b9d899a" containerName="maas-api"
Sep 10 21:33:31.370122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.370061    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4525225-3e29-4e5f-b64e-d1c65b9d899a" containerName="maas-api"
Sep 10 21:33:31.370122 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.370114    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4525225-3e29-4e5f-b64e-d1c65b9d899a" containerName="maas-api"
Sep 10 21:33:31.373282 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.373260    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.375553 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.375533    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jkhl9\"/\"kube-root-ca.crt\""
Sep 10 21:33:31.375675 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.375536    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jkhl9\"/\"openshift-service-ca.crt\""
Sep 10 21:33:31.376405 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.376353    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jkhl9\"/\"default-dockercfg-n4mbj\""
Sep 10 21:33:31.382210 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.382191    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"]
Sep 10 21:33:31.480013 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.479985    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpz7l\" (UniqueName: \"kubernetes.io/projected/023b0073-3b16-455c-a4b5-6b5e470286a0-kube-api-access-rpz7l\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.480176 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.480019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-lib-modules\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.480176 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.480038    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-proc\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.480176 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.480121    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-podres\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.480176 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.480142    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-sys\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581206 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581172    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-podres\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581371 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581213    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-sys\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581371 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581256    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpz7l\" (UniqueName: \"kubernetes.io/projected/023b0073-3b16-455c-a4b5-6b5e470286a0-kube-api-access-rpz7l\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581371 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581289    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-lib-modules\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581371 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-proc\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581371 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581355    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-podres\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581609 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581355    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-sys\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581609 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-lib-modules\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.581609 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.581465    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/023b0073-3b16-455c-a4b5-6b5e470286a0-proc\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.589426 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.589409    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpz7l\" (UniqueName: \"kubernetes.io/projected/023b0073-3b16-455c-a4b5-6b5e470286a0-kube-api-access-rpz7l\") pod \"perf-node-gather-daemonset-xggxr\" (UID: \"023b0073-3b16-455c-a4b5-6b5e470286a0\") " pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.684625 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.684543    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:31.954500 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:31.954425    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"]
Sep 10 21:33:31.958160 ip-10-0-140-211 kubenswrapper[2557]: W0910 21:33:31.958129    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod023b0073_3b16_455c_a4b5_6b5e470286a0.slice/crio-11d0ddd62b085605382983f66786a4cfcf327ac7b737cba0da190a9154d19386 WatchSource:0}: Error finding container 11d0ddd62b085605382983f66786a4cfcf327ac7b737cba0da190a9154d19386: Status 404 returned error can't find the container with id 11d0ddd62b085605382983f66786a4cfcf327ac7b737cba0da190a9154d19386
Sep 10 21:33:32.075099 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:32.075073    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr" event={"ID":"023b0073-3b16-455c-a4b5-6b5e470286a0","Type":"ContainerStarted","Data":"11d0ddd62b085605382983f66786a4cfcf327ac7b737cba0da190a9154d19386"}
Sep 10 21:33:32.135790 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:32.135766    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-jh6c8_a44413b6-44a3-4b54-ad71-14348181926e/download-server/0.log"
Sep 10 21:33:33.081920 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:33.081887    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr" event={"ID":"023b0073-3b16-455c-a4b5-6b5e470286a0","Type":"ContainerStarted","Data":"de2582a045dc85c1cc788bc03601b6e73bcffa9b681081d257c533b36814c29d"}
Sep 10 21:33:33.082305 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:33.081969    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:33.100422 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:33.100373    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr" podStartSLOduration=2.1003597960000002 podStartE2EDuration="2.100359796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:33.098531283 +0000 UTC m=+2093.117936600" watchObservedRunningTime="2026-09-10 21:33:33.100359796 +0000 UTC m=+2093.119765113"
Sep 10 21:33:33.983060 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:33.983029    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r6rnh_f171cba7-5c57-4da6-a497-750d801b1ce2/dns/0.log"
Sep 10 21:33:34.005187 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:34.005156    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r6rnh_f171cba7-5c57-4da6-a497-750d801b1ce2/kube-rbac-proxy/0.log"
Sep 10 21:33:34.130236 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:34.130201    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z7b7f_ddab8ca1-5c10-41d1-8b83-5cf0ab36138a/dns-node-resolver/0.log"
Sep 10 21:33:34.679616 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:34.679495    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-58c866cf74-f6sw9_73c3a136-cde8-4373-a1ef-01124e302c3b/registry/0.log"
Sep 10 21:33:34.729335 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:34.729303    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bn477_ff2e3ab3-8f3a-4d95-9734-6c33367725fb/node-ca/0.log"
Sep 10 21:33:35.992538 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:35.992506    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5tts4_ae9a0726-e680-4910-a364-c5f46d94210b/kube-auth-proxy/0.log"
Sep 10 21:33:36.128470 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:36.128433    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-p7hvp_6d437794-4d5d-49d5-8005-31a16c2ee4c3/payload-processing/0.log"
Sep 10 21:33:37.025716 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:37.025672    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mt2bd_a483da08-36c2-4bb6-b1c8-495b59ef3d61/serve-healthcheck-canary/0.log"
Sep 10 21:33:37.661522 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:37.661495    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfvm7_04bdb162-6a86-43fc-a5b7-f3de85e4b1e5/kube-rbac-proxy/0.log"
Sep 10 21:33:37.682974 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:37.682944    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfvm7_04bdb162-6a86-43fc-a5b7-f3de85e4b1e5/exporter/0.log"
Sep 10 21:33:37.705674 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:37.705642    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cfvm7_04bdb162-6a86-43fc-a5b7-f3de85e4b1e5/extractor/0.log"
Sep 10 21:33:39.104168 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:39.104135    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jkhl9/perf-node-gather-daemonset-xggxr"
Sep 10 21:33:40.610811 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:40.610656    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:33:40.622040 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:40.622015    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-65xvx_8be13ce3-5ffd-4463-8aa2-4bc9b94440ad/ovn-acl-logging/0.log"
Sep 10 21:33:40.681076 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:40.681047    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-2xnnx_1d8de411-a529-4246-9c4c-309d9bdb4e07/manager/0.log"
Sep 10 21:33:40.701726 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:40.701679    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55d69f6bb5-2xnnx_1d8de411-a529-4246-9c4c-309d9bdb4e07/copy-manifests/0.log"
Sep 10 21:33:42.147602 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:42.147567    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-79fcf6dff7-6dm5x_814d335d-d557-46cd-ba4c-ace8e839d7b7/manager/0.log"
Sep 10 21:33:42.200884 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:42.200851    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-k7g45_10c49916-fcf0-4c69-b821-9dcf4e2f4899/openshift-lws-operator/0.log"
Sep 10 21:33:42.385000 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:42.384974    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5tts4_ae9a0726-e680-4910-a364-c5f46d94210b/kube-auth-proxy/0.log"
Sep 10 21:33:42.498153 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:42.498121    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-p7hvp_6d437794-4d5d-49d5-8005-31a16c2ee4c3/payload-processing/0.log"
Sep 10 21:33:44.251884 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:44.251854    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-cd8d54847-5tts4_ae9a0726-e680-4910-a364-c5f46d94210b/kube-auth-proxy/0.log"
Sep 10 21:33:44.366295 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:44.366257    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-p7hvp_6d437794-4d5d-49d5-8005-31a16c2ee4c3/payload-processing/0.log"
Sep 10 21:33:47.376838 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:47.376806    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-f9d28_c5aec7f5-66d7-4f47-980b-981e395d58a8/migrator/0.log"
Sep 10 21:33:47.400175 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:47.400152    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-f9d28_c5aec7f5-66d7-4f47-980b-981e395d58a8/graceful-termination/0.log"
Sep 10 21:33:47.787652 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:47.787610    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-sjd8s_aa136b74-710f-492b-9b2b-b24fdc0029a6/kube-storage-version-migrator-operator/1.log"
Sep 10 21:33:47.789113 ip-10-0-140-211 kubenswrapper[2557]: I0910 21:33:47.789090    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-sjd8s_aa136b74-710f-492b-9b2b-b24fdc0029a6/kube-storage-version-migrator-operator/0.log"